output_allreduce/0000700000175100017500000000000014773363152014633 5ustar jenkinsHwHiAiUseroutput_allreduce/worker_3.log0000644000175100017500000715202314773362776017130 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:26.853.763 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:26.869.020 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 3 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.498 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.611 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.650 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.673 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.693 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.712 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.730 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.747 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.762 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.779 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.794 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.810 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.827 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.843 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.859 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.878 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.895 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.913 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.928 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.942 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.959 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.253.996 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.011 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.031 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.048 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.062 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.094 [ir_data_type_symbol_store.cc:240]619431 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.117 [ir_data_type_symbol_store.cc:240]619431 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.132 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.146 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.160 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.178 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.203 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.220 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.235 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.251 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.265 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.282 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.294 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.310 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.327 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.342 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.358 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.381 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.394 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.410 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.424 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.439 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.453 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.466 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.483 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.499 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.514 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.528 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.541 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.557 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.574 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.590 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.605 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.624 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.639 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.653 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.669 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.686 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.699 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.715 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.729 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.745 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.769 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.785 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.799 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.817 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.829 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.844 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.861 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.877 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.891 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.908 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.920 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.932 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.946 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.962 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.976 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.254.992 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.008 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.022 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.036 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.053 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.066 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.082 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.097 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.110 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.126 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.140 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.160 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.177 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.193 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.204 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.218 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.234 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.248 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.263 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.276 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.293 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.308 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.322 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.340 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.354 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.370 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.383 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.399 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.414 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.430 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.444 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.460 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.476 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.498 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.513 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.529 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.554 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.571 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.587 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.602 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.616 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.630 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.648 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.661 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.677 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.690 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.704 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.721 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.736 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.749 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.765 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.779 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.796 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.811 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.826 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.840 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.856 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.870 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.887 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.900 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.914 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.929 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.953 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.972 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.255.988 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.002 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.018 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.032 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.047 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.060 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.077 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.090 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.107 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.120 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.145 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.178 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.198 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.215 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.232 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.247 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.262 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.276 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.291 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.306 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.319 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.335 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.359 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.372 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.385 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.400 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.418 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.431 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.447 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.464 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.477 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.494 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.511 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.525 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.539 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.559 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.585 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.602 [ir_data_type_symbol_store.cc:173]619431 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.617 [ir_data_type_symbol_store.cc:219]619431 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619431,python3.9):2025-04-03-09:35:27.256.635 [ir_data_type_symbol_store.cc:182]619431 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.256 [awatchdog_core.c:99](tid:619431) watchdog get platform host side [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.410 [awatchdog_monitor.c:127](tid:619431) create monitor thread successfully, tid :281472636280736 [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.431 [awatchdog_monitor.c:226](tid:619431) AwdMonitorInit [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.613 [trace_attr.c:99](tid:619431) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.644 [trace_attr.c:189](tid:619431) attr init success, timeout=0ms. [WARNING] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.659 [trace_system_api.c:115](tid:619431) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.714 [trace_recorder.c:40](tid:619431) home_directory=/home/jenkins. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.732 [trace_recorder.c:121](tid:619431) use root path: /home/jenkins/ascend [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.746 [trace_event.c:151](tid:619431) init trace event success [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.263.801 [atrace_client_core.c:185](tid:619431) atrace client init success. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.102 [stacktrace_dumper.c:218](tid:619431) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.128 [trace_system_api.c:115](tid:619431) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.223 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 2 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.241 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 15 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.253 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 3 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.266 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 4 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.280 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 5 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.291 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 6 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.303 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 7 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.313 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 8 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.326 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 11 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.337 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 24 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.351 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 25 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.365 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 31 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.379 [stacktrace_signal.c:165](tid:619431) register signal handler for signo 35 succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.391 [stacktrace_signal.c:168](tid:619431) trace signal init succeed. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:27.265.401 [trace_core.c:51](tid:619431) trace init success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.266.079 [task_fail_callback_data_manager.cc:58] 619431 TaskFailCallBackManager: Constructor. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.275.725 [operator_factory_impl.cc:232]619431 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619431,python3.9):2025-04-03-09:35:27.275.761 [operator_factory_impl.cc:246]619431 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619431,python3.9):2025-04-03-09:35:27.275.774 [operator_factory_impl.cc:239]619431 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619431,python3.9):2025-04-03-09:35:27.283.004 [adapter_rts.cc:2720] [619431][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.298.007 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.298.083 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:27.298.110 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:27.308.172 [prof_cann_plugin.cpp:256] (tid:619431) Register module[6] callback. [INFO] OP(619431,python3.9):2025-04-03-09:35:27.358.529 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619431] cachelimit is 10000 [INFO] OP(619431,python3.9):2025-04-03-09:35:27.362.658 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619431] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619431,python3.9):2025-04-03-09:35:27.362.714 [op_dfx.cpp:54][NNOP][SystemConfig][619431] enable debug kernel flag 0. [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:27.362.736 [prof_cann_plugin.cpp:256] (tid:619431) Register module[61] callback. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.416.110 [mem_manager.cc:138]619431 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.423.218 [cache_policy.cc:47]619431 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619431,python3.9):2025-04-03-09:35:27.437.980 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.114 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.218 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.248 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.271 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.293 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.315 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.336 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.358 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.378 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.401 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.423 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.441 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.463 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.484 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.506 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.526 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.550 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.573 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.594 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.699 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.818 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.843 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.937 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.964 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.438.987 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.009 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.030 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.051 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.073 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.095 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.117 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.139 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.160 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.180 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.204 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.224 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.247 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.268 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.291 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.313 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.333 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.429 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.530 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.639 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.739 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.837 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.864 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.439.962 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.072 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.096 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.214 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.325 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.350 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.376 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.398 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.420 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.440 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.464 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.487 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.509 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.532 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.553 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.575 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.597 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.621 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.716 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.815 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.440.911 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.008 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.034 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.056 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.078 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.193 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.294 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.319 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.415 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.440 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.533 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.630 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.737 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.441.835 [addn.cc:29]619431 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.195 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.301 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.396 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.487 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.576 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.669 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.768 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.861 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.878 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619431,python3.9):2025-04-03-09:35:27.442.969 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619431,python3.9):2025-04-03-09:35:27.443.062 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619431,python3.9):2025-04-03-09:35:27.443.150 [symbolic_kernel_factory.cc:27]619431 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619431,python3.9):2025-04-03-09:35:27.449.856 [process_node_engine_manager.cc:140]619431 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619431,python3.9):2025-04-03-09:35:27.449.977 [process_node_engine_manager.cc:140]619431 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619431,python3.9):2025-04-03-09:35:27.456.594 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619431,python3.9):2025-04-03-09:35:27.456.630 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619431,python3.9):2025-04-03-09:35:27.457.163 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619431] start packet count timer [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:27.478.922 [prof_atls_plugin.cpp:117] (tid:619431) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:27.478.963 [prof_atls_plugin.cpp:117] (tid:619431) RegisterProfileCallback, callback type is 8 [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.555.755 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.556.297 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.605.844 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.605.993 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 3 [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.606.038 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.606.902 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.598 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.680 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.720 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.816 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.849 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.907 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.608.934 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.610.650 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.610.681 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.610.722 [acl.cpp:753]619431 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.610.794 [runtime_keeper.cc:78] 619431 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.685 [plugin_old_arch.cc:16] 619431 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaae7332900 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.720 [runtime_keeper.cc:149] 619431 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.736 [runtime_keeper.cc:221] 619431 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.764 [runtime.cc:1387] 619431 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.793 [runtime.cc:1163] 619431 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.821 [npu_driver.cc:4062] 619431 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.842 [runtime.cc:5096] 619431 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.857 [runtime.cc:1173] 619431 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.868 [runtime.cc:1183] 619431 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.887 [runtime.cc:1193] 619431 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.902 [profiler.cc:84] 619431 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.926 [runtime.cc:1209] 619431 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.953 [runtime.cc:1219] 619431 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.965 [runtime.cc:1230] 619431 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.977 [runtime.cc:1242] 619431 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.614.990 [runtime.cc:1264] 619431 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.004 [runtime.cc:1270] 619431 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.117 [runtime.cc:109] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.545 [runtime.cc:121] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.621 [runtime.cc:1314] 619431 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.639 [runtime.cc:1322] 619431 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.760 [runtime.cc:1338] 619431 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.816 [utils.cc:105] 619431 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.867 [runtime.cc:1664] 619431 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.907 [runtime.cc:1688] 619431 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.929 [runtime.cc:1953] 619431 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.615.958 [runtime.cc:1477] 619431 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:27.615.980 [prof_atls_plugin.cpp:210] (tid:619431) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:27.616.001 [heterogenous.cc:19] 619431 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:27.616.030 [api_c_soc.cc:70] 619431 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.616.047 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.616.062 [acl.cpp:760]619431 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.616.093 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.654.159 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.688.285 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.688.739 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.689.189 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.690.003 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.690.441 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.692.216 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.692.267 [acl.cpp:753]619431 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.692.296 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:27.692.311 [acl.cpp:760]619431 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:27.692.336 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.613.587 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.615.377 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.635.191 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.643.848 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.646.837 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.649.923 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.677.666 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619431,python3.9):2025-04-03-09:35:28.692.693 [platform_info.cpp:853]619431 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.582.178 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.584.005 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.584.147 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.586.383 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.586.524 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.586.644 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.586.786 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.586.899 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.588.028 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.588.177 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.101 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.241 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.361 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.478 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.587 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619431,python3.9):2025-04-03-09:35:29.590.697 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619431,python3.9):2025-04-03-09:35:30.233.356 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619431,python3.9):2025-04-03-09:35:34.479.959 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.736.522 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.736.598 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.736.628 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619431,python3.9):2025-04-03-09:35:34.782.068 [ops_kernel_builder_registry.cc:55]619431 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619431,python3.9):2025-04-03-09:35:34.782.170 [ops_kernel_builder_registry.cc:28]619431 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.819.481 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.819.561 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.819.626 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.819.644 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:34.819.659 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619431,python3.9):2025-04-03-09:35:35.638.221 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:37.357.345 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:37.357.946 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:37.368.852 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.456.579 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:37.456.653 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.457.885 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.457.933 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.457.953 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.457.976 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.233 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471661563808 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.360 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471653109664 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.382 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.495 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471644655520 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.619 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471636201376 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.644 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.661 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.838 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58702, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.458.875 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [INFO] DISTRIBUTED(619431,ffff3964dfa0,python3.9):2025-04-03-09:35:37.458.895 [mindspore/ccsrc/distributed/rpc/tcp/connection.cc:239] ReconnectSourceSocket] Connection from 127.0.0.1:58702 to 127.0.0.1:8118 is not available yet, errno: 0 Success, events: 8220 [INFO] DISTRIBUTED(619431,ffff3964dfa0,python3.9):2025-04-03-09:35:37.458.942 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:35] DoDisconnect] Failed to call connect, fd: 20from : 127.0.0.1:58702 to: 127.0.0.1:8118, events: 8216, errno: 111 Connection refused [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:37.958.962 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:76] Connect] Failed to connect to the tcp server : 127.0.0.1:8118, retry to reconnect(1/1)... [INFO] DISTRIBUTED(619431,ffff3964dfa0,python3.9):2025-04-03-09:35:37.958.968 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 20 state is kDisconnecting. Start disconnecting from 127.0.0.1:58702 to 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.459.066 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:173] Register] Failed to connect to the meta server node url: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.459.087 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:363] ReconnectWithTimeoutWindow] Failed to register and try to reconnect to the meta server. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.959.160 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:386] Reconnect] Start reconnecting business client. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.959.179 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.959.290 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:46832, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:38.959.309 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619431,ffff3a66dfa0,python3.9):2025-04-03-09:35:38.959.327 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:46832 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.384 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.402 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.419 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:393] Reconnect] Start reconnecting heartbeat client. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.440 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.526 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 22 source: 127.0.0.1:46870, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619431,ffff3964dfa0,python3.9):2025-04-03-09:35:39.459.540 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:46870 to 127.0.0.1:8118 is successfully created. System errno: Success [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.459.544 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 3 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.959.633 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.959.651 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.959.667 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.959.682 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.959.902 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 3 has been registered successfully. [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:39.960.104 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619431,ffff2bffefa0,python3.9):2025-04-03-09:35:39.960.148 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619431,ffff2bffefa0,python3.9):2025-04-03-09:35:39.960.180 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.184 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3002.24 msec. [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.207 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.221 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.239 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 3 rank id: 3 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.253 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.277 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 8886 to 9141 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.297 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.072 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.322 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.351 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3002.51 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.377 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.488 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.505 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.528 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.556 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.570 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.460.584 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.462.263 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.462.282 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 8886 to 9141 [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.462.430 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:9e147118-0198-457d-93cf-ed98c5519429, the ip:127.0.0.1, the port:8886 [INFO] PS(619431,ffff2b7eefa0,python3.9):2025-04-03-09:35:40.462.449 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619431,ffff2b7eefa0,python3.9):2025-04-03-09:35:40.462.471 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.464.055 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.464.194 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_3 has been registered successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.464.218 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:40.464.319 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.760 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 3 has been started successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.792 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.837 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.867 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.892 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 3, global rank size: 8 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.918 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3004.51 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.464.935 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 3 is devserver-c233-2 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.019 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.073 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.093 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 3 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.108 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 3 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.132 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [3]. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.153 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.214 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.214 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.241 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.262 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.465.300 [device.cpp:365]619431 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.465.413 [api_impl.cc:249] 619431 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.465.430 [api_impl.cc:6795] 619431 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.465.442 [device.cpp:371]619431 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.465.501 [callback.cpp:37]619431 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.465.535 [callback.cpp:44]619431 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.558 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.465.589 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.465.623 [device.cpp:143]619431 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.465.724 [npu_driver.cc:124] 619431 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.465.747 [api_impl.cc:2999] 619431 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.465.780 [runtime.cc:3602] 619431 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.466.025 [hdc_core.c:1982][ascend][curpid:619431,619431][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4220; pid=619431) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.466.206 [runtime.cc:3609] 619431 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.466.248 [atrace_client_communication.c:103](tid:619431) send [hello] msg. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.466.264 [atrace_client_communication.c:63](tid:619431) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.466.377 [hdc_core.c:1982][ascend][curpid:619431,619431][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4221; pid=619431) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:43.466.392 [hdc_api.cpp:671][tid:619431] connect succ, peer_node: 0, peer_devid: 3 [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.467.732 [hdc_client.c:417][ascend][curpid:619431,619431][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4221) [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.467.767 [atrace_client_communication.c:65](tid:619431) send msg by adx[finish], ret=0. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.467.779 [atrace_client_communication.c:97](tid:619431) msgType=1, magic=49959, version=1. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.467.789 [atrace_client_communication.c:135](tid:619431) send [hello] msg success. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.467.858 [atrace_client_core.c:160](tid:619431) start to receive device[3] trace log. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.467.892 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.467.907 [runtime.cc:2855] 619431 startAicpuExecutor: TsdOpen devId=3, device mode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.467.919 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.467.962 [client_manager.cpp:160][GetPlatformInfo][tid:619431] [TsdClient] mode[1] chipType[5] [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.467.994 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619431] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.007 [client_manager.cpp:582][GetVisibleDevices][tid:619431] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.022 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.468.019 [hdc_core.c:1982][ascend][curpid:619431,619571][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4222; pid=619431) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.033 [client_manager.cpp:125][GetInstance][tid:619431] [ClientManager] GetInstance, deviceId[3], g_runningMode[0], begin saving instance [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:43.468.051 [hdc_api.cpp:671][tid:619571] connect succ, peer_node: 0, peer_devid: 3 [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.080 [client_manager.cpp:435][IsHostEnvironment][tid:619431] [TsdClient] logicDeviceId is [3], hostaicpunum[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.105 [client_manager.cpp:464][GetClientRunMode][tid:619431] runningMode:0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.117 [client_manager.cpp:127][GetInstance][tid:619431] [ClientManager] Current mode:1 [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.468.136 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619431] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.147 [client_manager.cpp:347][GetProfilingMode][tid:619431] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.468.193 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619431] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.207 [process_mode_base.cpp:149][GetLogLevel][tid:619431] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.468.313 [atrace_client_communication.c:177](tid:619571) send [start] msg. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.316 [process_mode_base.cpp:161][GetLogLevel][tid:619431] [TsdClient] Get Log Level[001] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.336 [process_mode_manager.cpp:71][OpenProcess][tid:619431] [ProcessModeManager] enter into open process deviceId[3] rankSize[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.350 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619431] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.468.363 [atrace_client_communication.c:189](tid:619571) send [start] msg success. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.424 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[.], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.496 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[scene.info], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.557 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.641 [client_manager.cpp:264][CheckPackageExists][tid:619431] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.655 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[ascend_install.info], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.716 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[version.info], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.774 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[..], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.833 [client_manager.cpp:261][CheckPackageExists][tid:619431] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.959 [client_manager.cpp:268][CheckPackageExists][tid:619431] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.973 [client_manager.cpp:283][CheckPackageExists][tid:619431] [TsdClient][deviceId=3] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.468.984 [client_manager.cpp:286][CheckPackageExists][tid:619431] [TsdClient][deviceId=3] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.469.051 [devdrv_hccl.c:103][ascend][curpid:619431,619431][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619431) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.074 [process_mode_manager.cpp:396][InitTsdClient][tid:619431] [TsdClient][deviceId=3] get process sign success, procPid[619431] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.087 [process_mode_manager.cpp:403][InitTsdClient][tid:619431] [TsdClient] deviceId[3] begin to init hdc client [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.122 [hdc_common.cpp:67][InitMsgSize][tid:619431] HdcCommon::InitMsgSize Start [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.135 [hdc_common.cpp:80][InitMsgSize][tid:619431] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.149 [hdc_client.cpp:184][CreateHdcSession][tid:619431] HdcClient::CreateHdcSession Start [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.231 [hdc_client.cpp:208][CreateHdcSession][tid:619431] [HdcClient] deviceId: 3 connect session [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.469.261 [hdc_core.c:1982][ascend][curpid:619431,619431][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4223; pid=619431) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.275 [hdc_client.cpp:219][CreateHdcSession][tid:619431] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.288 [hdc_client.cpp:230][CreateHdcSession][tid:619431] [HdcClient] deviceId: 3 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.304 [version_verify.cpp:36][SetVersionInfo][tid:619431] VersionVerify: send client version to server [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.353 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.366 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.379 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.469.418 [internal_api.cpp:101][GetScheduleEnv][tid:619431] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.795 [version_verify.cpp:68][PeerVersionCheck][tid:619431] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.814 [version_verify.cpp:89][ParseVersionInfo][tid:619431] VersionVerify: pass client version info success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.826 [hdc_client.cpp:275][CheckHdcConnection][tid:619431] Service[2] create hdc success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.844 [version_verify.cpp:122][SpecialFeatureCheck][tid:619431] VersionVerify: new type[35], supported [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.469.858 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619431] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.889 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619431] [TsdClient][deviceId=3] [sessionId=1] wait package info respond [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.469.902 [hdc_client.cpp:154][TsdRecvData][tid:619431] recv timeout is:150000 ms [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.894 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619431] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.908 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619431] [PackageInfoMsgProc] [sessionID=1] deviceId[3], checkCode[56684994] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.921 [hdc_client.cpp:172][TsdRecvData][tid:619431] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.934 [version_verify.cpp:122][SpecialFeatureCheck][tid:619431] VersionVerify: new type[64], supported [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.955 [hdc_client.cpp:411][Destroy][tid:619431] enter HdcClient::Destroy() function [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.470.968 [hdc_client.c:417][ascend][curpid:619431,619431][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4223) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.470.993 [hdc_client.cpp:394][DestroyClient][tid:619431] begin drvHdcClientDestroy [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.007 [hdc_client.cpp:399][DestroyClient][tid:619431] end drvHdcClientDestroy [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.018 [hdc_client.cpp:377][ClearClientPtr][tid:619431] begin HdcClient::ClearClientPtr [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.030 [hdc_client.cpp:385][ClearClientPtr][tid:619431] end HdcClient::ClearClientPtr [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.039 [hdc_client.cpp:418][Destroy][tid:619431] end HdcClient::Destroy() function [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.078 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619431] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.095 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619431] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.471.111 [devdrv_hccl.c:103][ascend][curpid:619431,619431][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619431) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.124 [process_mode_manager.cpp:396][InitTsdClient][tid:619431] [TsdClient][deviceId=3] get process sign success, procPid[619431] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.135 [process_mode_manager.cpp:403][InitTsdClient][tid:619431] [TsdClient] deviceId[3] begin to init hdc client [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.149 [hdc_common.cpp:67][InitMsgSize][tid:619431] HdcCommon::InitMsgSize Start [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.160 [hdc_common.cpp:80][InitMsgSize][tid:619431] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.173 [hdc_client.cpp:184][CreateHdcSession][tid:619431] HdcClient::CreateHdcSession Start [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.229 [hdc_client.cpp:208][CreateHdcSession][tid:619431] [HdcClient] deviceId: 3 connect session [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.471.256 [hdc_core.c:1982][ascend][curpid:619431,619431][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4224; pid=619431) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.269 [hdc_client.cpp:219][CreateHdcSession][tid:619431] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.281 [hdc_client.cpp:230][CreateHdcSession][tid:619431] [HdcClient] deviceId: 3 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.294 [version_verify.cpp:36][SetVersionInfo][tid:619431] VersionVerify: send client version to server [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.307 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.317 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.328 [version_verify.cpp:52][SetVersionInfo][tid:619431] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.590 [version_verify.cpp:68][PeerVersionCheck][tid:619431] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.613 [version_verify.cpp:89][ParseVersionInfo][tid:619431] VersionVerify: pass client version info success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.625 [hdc_client.cpp:275][CheckHdcConnection][tid:619431] Service[2] create hdc success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.639 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619431] [TsdClient] tsd get process sign successfully, procpid[619431] signSize[48] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.654 [version_verify.cpp:114][SpecialFeatureCheck][tid:619431] VersionVerify: previous type[6], supported [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.681 [process_mode_manager.cpp:88][OpenProcess][tid:619431] [ProcessModeManager] deviceId[3] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619431,python3.9):2025-04-03-09:35:43.471.696 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619431] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.471.707 [hdc_client.cpp:154][TsdRecvData][tid:619431] recv timeout is:150000 ms [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.672.986 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619431] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.673.003 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619431] [TsdClient] DeviceMsgProc recvMsg realDeviceId[3] msgType[8] localDevId[3] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.673.018 [hdc_client.cpp:172][TsdRecvData][tid:619431] [deviceId=3] the Tsd recv data already [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.673.046 [devmm_svm.c:132][ascend][curpid:619431,619431][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.674.996 [devmm_virt_interface.c:450][ascend][curpid:619431,619431][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.675.322 [drv_log_user.c:623][ascend][curpid:619431,619431][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=3; devid=3; vfid=0; hostpid=619431; devpid=31148) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.675.345 [devmm_svm.c:122][ascend][curpid:619431,619431][drv][devmm][devmm_set_device_info]Device info. (devid=3; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.675.616 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619431] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.675.634 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619431] enter into OpenInHost deviceid[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.675.658 [client_manager.cpp:435][IsHostEnvironment][tid:619431] [TsdClient] logicDeviceId is [3], hostaicpunum[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.675.670 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619431] host cpu not support [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.675.680 [process_mode_manager.cpp:121][OpenProcess][tid:619431] [TsdClient][deviceId=3] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.675.693 [runtime.cc:2879] 619431 startAicpuExecutor: userDeviceId=3, devId=3, tsdOpen ret=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.675.704 [runtime.cc:2891] 619431 startAicpuExecutor: TsdOpen success, devId=3, ret=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.675.718 [runtime.cc:3627] 619431 DeviceRetain: new dev ok id=3, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.675.730 [raw_device.cc:936] 619431 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.675.751 [npu_driver.cc:2968] 619431 DeviceOpen: Open device start, device_id=3, tsId=0. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.675.828 [trs_sqcq.c:458][ascend][curpid:619431,619431][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.676.313 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=6; num=2044) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.676.814 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=7; num=2044) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.065 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=10; num=1024) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.305 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=11; num=1024) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.774 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=12; num=2048) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.797 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=8; num=16) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.816 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=9; num=16) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.830 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=13; num=4) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.677.847 [trs_sqcq.c:394][ascend][curpid:619431,619431][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=3; tsId=0; id_type=14; num=10) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.679.901 [trs_dev_drv.c:302][ascend][curpid:619431,619431][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=3; fd=37; mmapFd=38) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.679.921 [devmm_svm.c:132][ascend][curpid:619431,619431][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.679.940 [devmm_svm.c:122][ascend][curpid:619431,619431][drv][devmm][devmm_set_device_info]Device info. (devid=3; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.680.026 [npu_driver.cc:3011] 619431 DeviceOpen: Open device success, device_id=3, tsId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.680.041 [raw_device.cc:292] 619431 Init: chipType:0x5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.680.080 [raw_device.cc:304] 619431 Init: isAddrFlat:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.680.096 [raw_device.cc:320] 619431 Init: GetDevInfo, device_id=3, hardwareVersion=0x503 [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.680.110 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.680.124 [npu_driver.cc:1578] 619431 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.097 [raw_device.cc:352] 619431 Init: Get phy info, phy chip id 3, phy die id 0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.116 [runtime.cc:2771] 619431 GetTsdQos: [TSD] start get qos, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.126 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.136 [tsd_client.cpp:189][TsdCapabilityGet][tid:619431] TsdCapabilityGet Begin. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.156 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.168 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.178 [process_mode_manager.cpp:1042][CapabilityGet][tid:619431] [ProcessModeManager] enter into CapabilityGet process deviceId[3] type[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.191 [process_mode_manager.cpp:386][InitTsdClient][tid:619431] [TsdClient] tsd client has been inited before [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.205 [version_verify.cpp:114][SpecialFeatureCheck][tid:619431] VersionVerify: previous type[38], supported [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.237 [process_mode_manager.cpp:1065][CapabilityGet][tid:619431] [TsdClient][deviceId=3] [sessionId=1] [type=0]send capability success [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.251 [hdc_client.cpp:154][TsdRecvData][tid:619431] recv timeout is:150000 ms [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.542 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619431] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.590 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619431] [TsdClient] PidQosMsgProc recvMsg realDeviceId[3] msgType[39] localDevId[3] rspCode[0],pidqos[1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.607 [hdc_client.cpp:172][TsdRecvData][tid:619431] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.681.624 [process_mode_manager.cpp:1070][CapabilityGet][tid:619431] [TsdClient][logicDeviceId_=3][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.643 [runtime.cc:2782] 619431 GetTsdQos: [TSD] get qos success, userDeviceId=3, devId=3, qos=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.665 [engine.cc:78] 619431 Engine: Constructor. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.678 [stars_engine.cc:51] 619431 StarsEngine: Constructor. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.690 [engine_factory.cc:26] 619431 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.705 [engine_factory.cc:38] 619431 CreateEngine: Engine init success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.718 [raw_device.cc:364] 619431 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.732 [spm_pool.cc:42] 619431 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.947 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.963 [spm_pool.cc:55] 619431 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.681.978 [raw_device.cc:380] 619431 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.681.996 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.682.012 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.682.024 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.682.039 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.689.442 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.689.488 [task_allocator.cc:46] 619431 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.689.523 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.689.536 [h2d_copy_mgr.cc:45] 619431 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.689.551 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.689.566 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.690.681 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.690.702 [task_allocator.cc:46] 619431 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.690.717 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.690.730 [h2d_copy_mgr.cc:45] 619431 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.690.741 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.690.755 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.567 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.588 [task_allocator.cc:46] 619431 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.601 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.613 [h2d_copy_mgr.cc:45] 619431 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.624 [uma_arg_loader.cc:86] 619431 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.637 [task_allocator.cc:46] 619431 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.659 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.691.671 [uma_arg_loader.cc:93] 619431 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.501 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.521 [uma_arg_loader.cc:102] 619431 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.535 [uma_arg_loader.cc:104] 619431 Init: ALLOC PCIE is support[1] [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.692.548 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.563 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.684 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.700 [h2d_copy_mgr.cc:45] 619431 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.724 [raw_device.cc:393] 619431 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.738 [raw_device.cc:398] 619431 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.751 [task_allocator.cc:46] 619431 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.692.932 [buffer_allocator.cc:61] 619431 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.118 [raw_device.cc:109] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.512 [raw_device.cc:121] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.528 [raw_device.cc:425] 619431 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.577 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.666 [stream.cc:2246] 619431 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.705 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.693.721 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.094 [npu_driver.cc:6125] 619431 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.147 [npu_driver.cc:6137] 619431 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1047, streamId=0, logicCq=1047, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.173 [npu_driver.cc:6189] 619431 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=0, logicCq=1047. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.192 [npu_driver.cc:5916] 619431 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f61ae000, addr=dffff61ae000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.207 [stream.cc:653] 619431 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.253 [stream.cc:276] 619431 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.268 [stream.cc:688] 619431 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=3, streamResId=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.285 [stream_state_callback_manager.cc:45] 619431 Notify: not reg stream state callback func! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.300 [raw_device.cc:593] 619431 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.318 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.336 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=16 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.350 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.431 [tsch_stream.cc:45] 619431 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=3 streamResId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.448 [raw_device.cc:1219] 619431 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.526 [osal.cc:309] 619431 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.678 [osal.cc:182] 619431 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.710 [osal.cc:309] 619431 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.777 [stars_engine.cc:1785] 619582 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.791 [osal.cc:182] 619431 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.816 [stars_engine.cc:201] 619583 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.828 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.694.855 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.694.874 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.694.886 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.906 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.960 [npu_driver.cc:6306] 619582 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.981 [stars_engine.cc:1855] 619582 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.694.994 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.153 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.187 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.228 [task_info.cc:1596] 619431 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.244 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.258 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.289 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.304 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.321 [stars_engine.cc:1122] 619431 SyncTask: Task Wait: device_id=3, stream_id=0, task_id=0, logicCqId=1047. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.371 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.386 [stars_engine.cc:1071] 619431 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.411 [stars_engine.cc:1017] 619431 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.429 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.451 [stars_engine.cc:1027] 619431 ProcReport: taskType=97. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.472 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.487 [stars_engine.cc:1570] 619431 ProcLogicCqReport: CQE warning, device_id=3, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.505 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.519 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.534 [task_info.cc:1614] 619431 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.547 [raw_device.cc:1305] 619431 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.574 [stars_engine.cc:1351] 619431 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.595 [npu_driver.cc:5618] 619431 GetSqEnable: deviceId=3, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.608 [stars_engine.cc:1365] 619431 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.622 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.636 [npu_driver.cc:5595] 619431 GetSqTail: deviceId=3, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.648 [npu_driver.cc:5406] 619431 SetSqHead: deviceId=3, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.667 [npu_driver.cc:5357] 619431 EnableSq: deviceId=3, tsId=0, sqId=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.684 [stars_engine.cc:1430] 619431 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.700 [npu_driver.cc:5618] 619431 GetSqEnable: deviceId=3, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.714 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.727 [raw_device.cc:1331] 619431 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.695.741 [raw_device.cc:610] 619431 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.027 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.088 [ringbuffer_maintain_task.cc:98] 619431 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x1680 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.107 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.121 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.141 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.155 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.170 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.210 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.268 [stars_engine.cc:622] 619431 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=1047, retCode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.284 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.298 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.311 [stream.cc:2886] 619431 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.324 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.348 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.362 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.377 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.395 [stars_engine.cc:437] 619431 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.410 [stream.cc:2886] 619431 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.457 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.474 [task_info.cc:111] 619431 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.489 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.502 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.518 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.531 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.546 [stars_engine.cc:1122] 619431 SyncTask: Task Wait: device_id=3, stream_id=0, task_id=2, logicCqId=1047. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.674 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.688 [stars_engine.cc:1071] 619431 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.701 [stars_engine.cc:1017] 619431 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.717 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.729 [stars_engine.cc:1027] 619431 ProcReport: taskType=29. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.740 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.753 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.765 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.794 [npu_driver.cc:5618] 619431 GetSqEnable: deviceId=3, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.809 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.701.821 [raw_device.cc:1232] 619431 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.701.858 [atrace_api.c:40](tid:619431) AtraceCreate start. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.702.980 [trace_rb_log.c:45](tid:619431) init ring buffer time finished, realTime 1743644143701914807, monotonicTime 4211291185711, freq 99997 kHz, minutesWest 480 minutes [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.702.997 [trace_rb_log.c:137](tid:619431) [RUNTIME_ATRACE_DEV3_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.703.013 [tracer_mgr_operate.c:66](tid:619431) create object RUNTIME_ATRACE_DEV3_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.703.027 [atrace_api.c:44](tid:619431) AtraceCreate end. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.703.068 [trace_event.c:70](tid:619431) limitedNum 1, processType 0 [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.703.082 [trace_event.c:372](tid:619431) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.097 [runtime.cc:3159] 619431 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619431,python3.9):2025-04-03-09:35:43.703.110 [client_manager.cpp:196][SetProfilingCallback][tid:619431] [TsdClient] set profiling callback success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.129 [runtime.cc:3364] 619431 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.157 [context.cc:109] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.232 [context.cc:121] 619431 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.246 [context.cc:276] 619431 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.308 [raw_device.cc:803] 619431 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.325 [npu_driver_dcache_lock.cpp:39] 619431 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.469 [raw_device.cc:528] 619431 AllocStackPhyBase: AllocStackPhyBase device_id=3, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.516 [raw_device.cc:232] 619431 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.539 [runtime.cc:3393] 619431 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.553 [runtime.cc:3560] 619431 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.571 [api_impl.cc:2987] 619431 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.585 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.600 [api_impl.cc:2992] 619431 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:43.703.621 [awatchdog_core.c:175](tid:619431) create watchdog for id : 65543, tid : 619431, timeout : 300s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.703.645 [api_impl.cc:3025] 619431 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.703.678 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.703.693 [acl.cpp:121]619431 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.406 [api_impl.cc:3476] 619431 GetDeviceInfo: get device info, deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.442 [api_impl.cc:3476] 619431 GetDeviceInfo: get device info, deviceId=3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.706.487 [device.cpp:133]619431 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 3, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.706.508 [device.cpp:154]619431 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.706.526 [context.cpp:87]619431 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.706.550 [context.cpp:98]619431 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.706.571 [stream.cpp:48]619431 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.596 [api_impl.cc:1120] 619431 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.652 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.688 [stream.cc:2246] 619431 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.725 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.706.740 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.046 [npu_driver.cc:6125] 619431 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.106 [npu_driver.cc:6137] 619431 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1048, streamId=2, logicCq=1048, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.128 [npu_driver.cc:6189] 619431 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=2, logicCq=1048. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.144 [npu_driver.cc:5916] 619431 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec669000, addr=dfffec669000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.159 [stream.cc:653] 619431 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.707.207 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.224 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.707.393 [stream.cc:291] 619431 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.068 [stream.cc:688] 619431 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=3, streamResId=65535 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.099 [stream_state_callback_manager.cc:45] 619431 Notify: not reg stream state callback func! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.115 [api_impl.cc:1162] 619431 StreamCreate: Succ, flags=1536, stream_id=2, context=aaaaf46ef9e0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.710.145 [stream.cpp:189]619431 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.164 [stream.cc:3161] 619431 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.710.177 [stream.cpp:195]619431 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.710.228 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaaf481af80. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.710.273 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.710.286 [stream.cpp:48]619431 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.304 [api_impl.cc:1120] 619431 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.336 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.369 [stream.cc:2246] 619431 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.403 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.418 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.585 [npu_driver.cc:6125] 619431 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.634 [npu_driver.cc:6137] 619431 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1049, streamId=3, logicCq=1049, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.655 [npu_driver.cc:6189] 619431 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=3, logicCq=1049. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.671 [npu_driver.cc:5916] 619431 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec648000, addr=dfffec648000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.685 [stream.cc:653] 619431 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619431,python3.9):2025-04-03-09:35:43.710.729 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.710.746 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.711.051 [stream.cc:291] 619431 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.711.670 [stream.cc:688] 619431 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=3, streamResId=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.711.689 [stream_state_callback_manager.cc:45] 619431 Notify: not reg stream state callback func! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.711.701 [api_impl.cc:1162] 619431 StreamCreate: Succ, flags=1536, stream_id=3, context=aaaaf46ef9e0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.711.716 [stream.cpp:189]619431 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.711.732 [stream.cc:3161] 619431 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.711.745 [stream.cpp:195]619431 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.711.790 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaaf494a360. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.711.969 [callback.cpp:25]619431 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471394574240. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.016 [npu_driver.cc:462] 619431 EventIdAlloc: event id alloc begin, device_id=3, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.027 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.068 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.081 [npu_driver.cc:483] 619431 EventIdAlloc: Event id alloc success, device_id=3, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.082 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.099 [event.cc:347] 619431 GenEventId: Event id alloc success, device_id=3, tsId=0, eventFlag=1, event_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.118 [subscribe.cc:54] 619431 AssignGroupId: give groupId=64 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.132 [npu_driver.cc:3775] 619431 SqCqAllocate: deviceId=3, tsId=0, groupId=64. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.176 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.200 [npu_driver.cc:3785] 619431 SqCqAllocate: alloc success, deviceId=3 sqId=789, cqId=789. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.217 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.220 [subscribe.cc:131] 619431 Insert: callback subscribe: groupId=64, threadId=281471394574240, devId=3, tsId=0, streamId=2, sqId=789, cqId=789 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.244 [callback.cpp:31]619431 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471394574240. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.253 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.712.274 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaaf481af80. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.385 [callback.cpp:25]619431 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471386120096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.406 [npu_driver.cc:462] 619431 EventIdAlloc: event id alloc begin, device_id=3, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.425 [npu_driver.cc:483] 619431 EventIdAlloc: Event id alloc success, device_id=3, ts_id=0, event_id=1, eventFlag=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.419 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.440 [event.cc:347] 619431 GenEventId: Event id alloc success, device_id=3, tsId=0, eventFlag=1, event_id=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.460 [subscribe.cc:54] 619431 AssignGroupId: give groupId=65 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.473 [npu_driver.cc:3775] 619431 SqCqAllocate: deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.502 [npu_driver.cc:3785] 619431 SqCqAllocate: alloc success, deviceId=3 sqId=790, cqId=790. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.521 [subscribe.cc:131] 619431 Insert: callback subscribe: groupId=65, threadId=281471386120096, devId=3, tsId=0, streamId=3, sqId=790, cqId=790 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.712.539 [callback.cpp:31]619431 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471386120096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.549 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.712.566 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaaf494a360. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.712.588 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.712.663 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.712.986 [npu_driver.cc:2616] 619431 MemGetInfoEx: memory info type=1, type=2, free=8864157696, hug_free=56245616640 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.037 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.111 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.153 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.180 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.201 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62093M, Reserved MOC size for Other Components(HCCL/rts/etc.):3891M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58202M. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.713.221 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.713.246 [event.cpp:260]619431 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.713.266 [api_impl.cc:5713] 619431 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.342 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.362 [timeout_set_task.cc:115] 619431 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.378 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.393 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.423 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.438 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.454 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.469 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.506 [stars_engine.cc:622] 619431 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=1047, retCode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.521 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.535 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.549 [stream.cc:2886] 619431 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.563 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.576 [stream.cc:2886] 619431 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.588 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.602 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.616 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.629 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.647 [stars_engine.cc:437] 619431 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.662 [stream.cc:2886] 619431 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.714.675 [event.cpp:262]619431 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.714.690 [event.cpp:269]619431 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.709 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.725 [timeout_set_task.cc:115] 619431 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.738 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.753 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.769 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.783 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.797 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.819 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.833 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.848 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.861 [stream.cc:2886] 619431 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.875 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.886 [stream.cc:2886] 619431 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.902 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.914 [stream.cc:2886] 619431 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.928 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.941 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.954 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.965 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.981 [stars_engine.cc:437] 619431 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.714.994 [stream.cc:2886] 619431 StarsWaitForTask: No.4 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.006 [event.cpp:277]619431 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.033 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.081 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.099 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 3 has been set [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.117 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.132 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.173 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.196 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:43.715.242 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.341 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.715.370 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.400 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.433 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:43.715.450 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.464 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.609 [acl.cpp:288]619431 aclInit: start to execute aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.641 [acl.cpp:298]619431 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.715.733 [acl.cpp:242]619431 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.724.575 [error_manager.cc:673]619431 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:43.725.330 [acl.cpp:315]619431 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:43.725.403 [prof_atls_plugin.cpp:210] (tid:619431) Module[45] register callback of ctrl handle. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.725.439 [ge_executor.cc:273]619431 Initialize:Init GeExecutor begin. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.725.497 [op_lib_register.cc:117]619431 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.574 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.610 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.642 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.674 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.703 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.743 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.773 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.802 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.830 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.859 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.887 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.916 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.945 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.725.973 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.002 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.030 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.059 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.087 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.115 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.151 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.184 [op_lib_register.cc:165]619431 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.220 [plugin_manager.cc:616]619431 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.235 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.266 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.287 [plugin_manager.cc:624]619431 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.302 [plugin_manager.cc:206]619431 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.736 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.782 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.798 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.726.814 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.726.858 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.204 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.239 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.254 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.267 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.305 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.342 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.358 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.371 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.387 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.403 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.729 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.762 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.777 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.727.791 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.727.828 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.152 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.202 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.218 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.244 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.279 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.314 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.329 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.344 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.358 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.373 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.697 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.730 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.745 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.728.759 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.728.790 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.113 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.146 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.161 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.176 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.211 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.244 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.271 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.285 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.301 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.316 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.642 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.676 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.692 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.729.708 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.729.743 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.065 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.099 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.113 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.128 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.161 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.196 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.210 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.225 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.239 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.271 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.594 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.627 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.641 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.730.656 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.730.691 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.013 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.044 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.059 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.073 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.106 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.140 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.155 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.170 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.185 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.197 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.209 [plugin_manager.cc:218]619431 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.240 [plugin_manager.cc:422]619431 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.253 [plugin_manager.cc:155]619431 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.273 [plugin_manager.cc:158]619431 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.287 [plugin_manager.cc:426]619431 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.608 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.643 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.657 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.731.672 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.706 [plugin_manager.cc:439]619431 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.725 [plugin_manager.cc:175]619431 ReversePathString:Enter ReversePathString schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.731.745 [plugin_manager.cc:187]619431 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619431,python3.9):2025-04-03-09:35:43.732.070 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:43.732.101 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:43.732.121 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.732.176 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619431,python3.9):2025-04-03-09:35:43.752.941 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.753.639 [op_tiling_registry.cc:162]619431 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619431,python3.9):2025-04-03-09:35:43.753.767 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.753.874 [op_tiling_registry.cc:162]619431 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619431,python3.9):2025-04-03-09:35:43.753.989 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.098 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.201 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.321 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.337 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.356 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.458 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.562 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.663 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.772 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619431,python3.9):2025-04-03-09:35:43.754.880 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.248 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.345 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.458 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.567 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.683 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.788 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619431,python3.9):2025-04-03-09:35:43.755.901 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.090 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.207 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.316 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.420 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.435 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.540 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.651 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.666 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.762 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.864 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619431,python3.9):2025-04-03-09:35:43.756.966 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.757.175 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619431,python3.9):2025-04-03-09:35:43.757.350 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.757.448 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.757.552 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619431,python3.9):2025-04-03-09:35:43.757.661 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.106 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.222 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.343 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.524 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.649 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.771 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.879 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619431,python3.9):2025-04-03-09:35:43.758.983 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.082 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.187 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.303 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.405 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.421 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.529 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.637 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.740 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.755 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.853 [op_tiling_registry.cc:122]619431 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619431,python3.9):2025-04-03-09:35:43.759.954 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.065 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.174 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.292 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.493 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.580 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.675 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.751 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.824 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.760.989 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.108 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.134 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.156 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.177 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.379 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.458 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.619 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.702 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.799 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.824 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.846 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.761.919 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.008 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.083 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.160 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.234 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.309 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.387 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.485 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.565 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.639 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.732 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.900 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.762.977 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.763.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.763.134 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.763.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.767.362 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619431,python3.9):2025-04-03-09:35:43.767.393 [op_tiling_registry.cc:148]619431 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619431,python3.9):2025-04-03-09:35:43.768.868 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.768.999 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.408 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.656 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.773 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.769.927 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.041 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.200 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.352 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.478 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.593 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.728 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.852 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.969 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.770.995 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.156 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.269 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.412 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.535 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.680 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.771.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.072 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.210 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.326 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.448 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.578 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.772.924 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.047 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.170 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.197 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.415 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.536 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.684 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.880 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.773.998 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.208 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.371 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.483 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.627 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.653 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.762 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.774.901 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.019 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.129 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.240 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.356 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.577 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.605 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.775.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.177 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.420 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.776.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.077 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.242 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.376 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.492 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.611 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.724 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.777.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.291 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.332 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff05345d70, registered count:1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.501 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.523 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff0534e248, registered count:2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.769 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.778.926 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.040 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.170 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.404 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.518 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.754 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.779.896 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.006 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.257 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.381 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.622 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.890 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.918 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.942 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.780.965 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.118 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.237 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.363 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.480 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.596 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.733 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.756 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.881 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.781.992 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.107 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.306 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.423 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.536 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.654 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.778 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.782.896 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.013 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.040 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.159 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.270 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.299 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.585 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.813 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.783.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.194 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.312 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.422 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.626 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.748 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.784.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.410 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.454 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.479 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.616 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.739 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.860 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.787.971 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.087 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.332 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.444 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.569 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.597 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.725 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.752 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.888 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.788.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.141 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.257 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.607 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.789.895 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.790.052 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.790.169 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.790.295 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.790.416 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.791.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.791.897 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.792.241 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.792.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.793.324 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.793.450 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.793.571 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.793.724 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.793.846 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.794.143 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.796.372 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.797.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.797.408 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.797.568 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.798.580 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.798.736 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.798.899 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.390 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.636 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.869 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.799.994 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.113 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.247 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.365 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.481 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.612 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.800.892 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.012 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.197 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.322 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.437 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.562 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.714 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.801.895 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.017 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.144 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.172 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.359 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.776 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.802.896 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.176 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.419 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.447 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.564 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.591 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.701 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.818 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.844 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.803.967 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.085 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.226 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.411 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.740 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.804.893 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.073 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.635 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.759 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.805.893 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.011 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.151 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.339 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.456 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.578 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.695 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.813 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.806.929 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.807.052 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.807.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.807.205 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.807.442 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.808.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.811.163 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.811.366 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.811.578 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.811.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.812.088 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.812.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.096 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.262 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.400 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.555 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.716 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.858 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.813.999 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.143 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.278 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.401 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.515 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.602 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.901 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.929 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.814.988 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.009 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.031 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.052 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.077 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.161 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.184 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.207 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.227 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.248 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.268 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.289 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.315 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.336 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.356 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.377 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.396 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.417 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.458 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.550 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.573 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.596 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.619 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.642 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.663 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.683 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.705 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.747 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.778 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.823 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.844 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.868 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.889 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.909 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.934 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.956 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.815.982 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.003 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.025 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.047 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.067 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.087 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.108 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.129 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.288 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.435 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.631 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.654 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.677 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.816.950 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.171 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.278 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.386 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.541 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.651 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.817.933 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.050 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.079 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.105 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.220 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.245 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.270 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.374 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.525 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.683 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.830 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.818.939 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.237 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.422 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.533 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.645 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.672 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.696 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.819.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.013 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.168 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.331 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.514 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.662 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.775 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.820.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.078 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.255 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.380 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.490 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.608 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.721 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.836 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.821.985 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.247 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.358 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.591 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.822.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.130 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.676 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.789 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.827 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.823.945 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.057 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.181 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.296 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.407 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.516 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.631 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.824.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.002 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.118 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.305 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.423 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.545 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.657 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.825.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.016 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.133 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.242 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.359 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.472 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.581 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.707 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.826.966 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.827.079 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.827.238 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.827.480 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.827.669 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.827.838 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.011 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.228 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.447 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.664 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.712 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.732 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.763 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.787 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.810 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.828.831 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.829.544 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.829.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.830.358 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.830.564 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.830.766 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.831.149 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.831.277 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.831.442 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.834.123 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.835.904 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.043 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.178 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.295 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.574 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.729 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.836.910 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.298 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.322 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.343 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.520 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.549 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.707 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.837.872 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.043 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.398 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.584 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.745 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.838.911 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.088 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.257 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.443 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.586 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.772 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.804 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.909 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619431,python3.9):2025-04-03-09:35:43.839.940 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.057 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.087 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.219 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.250 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.363 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.619 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.740 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.840.952 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.841.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.841.342 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.841.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.842.076 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.842.285 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.842.629 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.842.832 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.027 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.144 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.179 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.294 [op_tiling_registry.cc:134]619431 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.323 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.514 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.541 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.843.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.844.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.293 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.471 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.632 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.667 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.845.971 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.130 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.162 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.329 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.486 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.634 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.797 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.846.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.847.341 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.847.687 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.847.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.848.114 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.849.175 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.849.589 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.291 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.447 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.566 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.724 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.881 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.850.917 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.025 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.054 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.373 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.486 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.797 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.851.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.218 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.408 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.555 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.670 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.852.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.114 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.315 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.505 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.661 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.853.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.855.358 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.860.997 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.861.049 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.863.789 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.864.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.864.347 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.864.677 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.864.881 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.252 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.418 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.612 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.755 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.865.848 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.866.105 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.866.235 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.866.391 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.866.888 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.219 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.251 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.367 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.500 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.682 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.867.884 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.868.071 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.868.236 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.868.352 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.868.465 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.868.493 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.162 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.535 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.817 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.845 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.869.994 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.119 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.146 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.171 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.326 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.531 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.621 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.870.864 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.871.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.871.079 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.871.113 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.871.300 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.871.331 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.149 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.264 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.286 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.358 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.440 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.460 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.540 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.708 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.725 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.841 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.861 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.878 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.876.896 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.877.065 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.877.084 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.878.224 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff05c98a10, registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.878.259 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff05c98c70, registered count:1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.878.679 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.879.156 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.880.820 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.880.993 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.881.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.881.460 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.881.898 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.190 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.325 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.514 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.594 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.734 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.913.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.914.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.914.610 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.914.871 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.914.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.915.131 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.915.375 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.915.554 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.915.963 [op_tiling_manager.cc:79]619431 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.035 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.220 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.286 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.344 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.410 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.450 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.492 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.916.533 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.376 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.414 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.434 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.452 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.472 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.781 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff384c59b0, registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.806 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.826 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.856 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.917.875 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.171 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff384c8da0, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.195 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.213 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.232 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.249 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.918.524 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.918.579 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.918.664 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.356 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff3847f2c0, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.391 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.407 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.421 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.435 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.741 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff38483978, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.766 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.792 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.806 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.919.820 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.920.087 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.137 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.218 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.270 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.308 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.348 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.920.380 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.416 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.452 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.920.492 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.528 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.563 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.599 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.635 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.672 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.920.701 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.736 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.777 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.920.809 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.844 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.882 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.927 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.964 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.920.999 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.029 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.064 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.101 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.130 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.165 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.202 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.237 [op_tiling_manager.cc:73]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.281 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.319 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.350 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.386 [op_tiling_manager.cc:58]619431 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.921.422 [op_tiling_manager.cc:67]619431 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.451 [op_tiling_manager.cc:75]619431 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.467 [op_tiling_manager.cc:109]619431 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [195248] micro seconds. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.579 [plugin_manager.cc:639]619431 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.597 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.622 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:43.921.683 [host_cpu_engine.cc:254]619431 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619431,python3.9):2025-04-03-09:35:43.926.639 [host_cpu_engine.cc:273]619431 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619431,python3.9):2025-04-03-09:35:43.926.713 [host_cpu_engine.cc:254]619431 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619431,python3.9):2025-04-03-09:35:43.932.178 [host_cpu_engine.cc:267]619431 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619431,python3.9):2025-04-03-09:35:43.932.942 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619431] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.932.988 [folding.cc:404][InitCpuConstantFoldingNew][tid:619431]:"Init cpu constant folding begin." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.026 [folding.cc:408][InitCpuConstantFoldingNew][tid:619431]:"Number of registered ops: 234" [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.040 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Acos]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.055 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Acosh]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.068 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.079 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.090 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.100 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.111 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.122 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Add]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.136 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Addcdiv]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.151 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Addcmul]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.161 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[AffineGrid]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.174 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ApplyAdamV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.184 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Asin]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.196 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Asinh]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.209 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Assert]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.221 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Atan]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.236 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Atan2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.248 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Atanh]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.260 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[AvgPoolV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.272 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Betainc]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.287 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Bucketize]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.299 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CTCLossV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.315 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.325 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CacheSwapTable]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.340 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.352 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CaseCondition]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.371 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cast]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.383 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Ceil]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.396 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CheckNumerics]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.407 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ClipByValue]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.420 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ClipByValueV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.431 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Combinations]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.443 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CompareAndBitpack]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.455 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.467 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ConcatV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Conj]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.489 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ConjugateTranspose]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.500 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.512 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cosh]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.523 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[CropAndResize]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.534 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cross]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.545 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cummax]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.558 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cummin]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.569 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Cumsum]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.581 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Div]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.593 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[DynamicStitch]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.604 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EditDistance]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.616 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Elu]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.627 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EluGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.638 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.649 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.660 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.675 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.685 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.701 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.713 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.727 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.740 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.752 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.764 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.776 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.787 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.798 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.809 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingLookup]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.823 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.836 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.849 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.860 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.872 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.884 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Equal]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.894 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Expand]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.905 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ExpandDims]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.917 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Exponential]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.928 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.939 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Fill]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.950 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[FillDiagonal]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.962 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Fills]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.974 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Floor]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.987 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[FloorDiv]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.933.999 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[FloorMod]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.012 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GEMM]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.024 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GatherElements]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.039 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GatherV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.050 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Geometric]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.060 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GetDynamicDims]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.072 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Greater]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.083 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GreaterEqual]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.096 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GridSampler2D]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.108 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.121 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GridSampler3D]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.132 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.143 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Histogram]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.155 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.167 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Identity]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.178 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[IndexToAddr]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.189 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.201 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InitPartitionMap]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.212 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.224 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.236 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InvGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.248 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[InvertPermutation]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.259 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[IsClose]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.270 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[IsInf]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.282 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[IsNan]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.295 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Less]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.307 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[LinSpace]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.319 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[LinearSumAssignment]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.331 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Log]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.343 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Log1p]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.366 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.390 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.403 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MaskedFill]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MaskedScatter]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.430 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MaskedSelect]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.440 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.452 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MatMul]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.464 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.477 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MatrixDiagV3]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.492 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MinAreaPolygons]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Mul]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.517 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.529 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.541 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.553 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NLLLoss]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.566 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NLLLossGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.578 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NMSWithMask]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.590 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Neg]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.601 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.613 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NonZero]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.625 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.637 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.649 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[NotEqual]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[OnesLike]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.674 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[PSAMask]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.687 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[PSAMaskGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.699 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Pad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.710 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[PadD]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.722 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[PadV3]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.738 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[PadV3Grad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.750 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Pinverse]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.763 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Poisson]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.775 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Pow]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.787 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.799 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RNNTLoss]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.812 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RandomStandardNormal]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.824 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RandomUniform]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.838 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RealDiv]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.849 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ReduceAll]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.861 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ReduceMean]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.873 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ReduceSum]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.890 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Reshape]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.903 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Resize]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.914 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ResizeBilinear]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.926 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.939 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.951 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ResizeGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.964 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ResizeTrilinear]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.977 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ReverseSequence]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.934.989 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RightShift]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.002 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.016 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[RotatedNMS]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.029 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Round]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.041 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Rsqrt]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.053 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[STFT]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.068 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.082 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ScatterElements]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.094 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ScatterNdAdd]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.112 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ScatterNdMax]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.125 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ScatterNdMin]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.137 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SearchSorted]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.149 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SelectV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.160 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Sigmoid]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.173 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SigmoidGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.184 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Sign]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.195 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SilentCheck]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.207 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SilentCheckV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.218 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SilentCheckV3]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SliceWrite]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.242 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SoftmaxV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.256 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Sort]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.270 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SparseSegmentMean]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.297 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.310 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SparseSegmentSum]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.322 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SparseToDense]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.333 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SpatialTransformer]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.345 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SplitD]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SplitV]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.366 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Sqrt]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.377 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SqrtGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.387 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Square]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.399 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[SquaredDifference]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.411 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[StatelessBernoulli]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.425 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.437 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[StatelessRandperm]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.457 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[StridedSlice]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.470 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[StridedSliceV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.485 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Sub]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.500 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TableToResource]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.511 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Tan]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.523 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TanhGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.535 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TensorEqual]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.549 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.561 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ThreeNN]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.575 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Tile]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TileWithAxis]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.597 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TopK]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.607 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TopKPQDistance]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.619 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.632 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TopKV2D]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Trace]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.656 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TransData]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.668 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[TransDataRNN]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.680 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Tril]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.692 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Triu]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.704 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.719 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.732 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UninitPartitionMap]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.744 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UniqueConsecutive]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.756 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.768 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.781 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.795 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.812 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.826 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.839 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ViewCopy]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.851 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Voxelization]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.862 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[WarpAffine]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.873 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[ZerosLike]." [INFO] AICPU(619431,python3.9):2025-04-03-09:35:43.935.885 [folding.cc:413][InitCpuConstantFoldingNew][tid:619431]:"Register op[Zeta]." [INFO] OP(619431,python3.9):2025-04-03-09:35:43.935.900 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619431] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619431,python3.9):2025-04-03-09:35:43.935.913 [host_cpu_engine.cc:273]619431 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619431,python3.9):2025-04-03-09:35:43.936.043 [plugin_manager.cc:872]619431 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.939.097 [plugin_manager.cc:883]619431 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.164 [ops_kernel_executor_manager.cc:37]619431 Initialize:No library to load [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.182 [plugin_manager.cc:540]619431 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.197 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.231 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.250 [plugin_manager.cc:547]619431 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.265 [plugin_manager.cc:206]619431 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.858 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.939.904 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.939.918 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.939.942 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.939.987 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.316 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.358 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.373 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.387 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.421 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.460 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.476 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.488 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.501 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.532 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.546 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.558 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.569 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.883 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.920 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.935 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.940.961 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.940.995 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.312 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.342 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.355 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.368 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.400 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.432 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.446 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.458 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.470 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.498 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.510 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.520 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.531 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.839 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.872 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.898 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.941.909 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.941.942 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.251 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.283 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.296 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.308 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.338 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.374 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.386 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.399 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.413 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.439 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.453 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.463 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.475 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.788 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.818 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.847 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.942.859 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.942.891 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.202 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.233 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.248 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.260 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.291 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.325 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.339 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.352 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.362 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.390 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.402 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.416 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.428 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.736 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.768 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.791 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.943.803 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.943.840 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.166 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.201 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.215 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.229 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.264 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.296 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.308 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.318 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.329 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.355 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.370 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.380 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.392 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.401 [plugin_manager.cc:218]619431 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.432 [plugin_manager.cc:422]619431 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.444 [plugin_manager.cc:155]619431 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.465 [plugin_manager.cc:158]619431 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.476 [plugin_manager.cc:426]619431 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.790 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.824 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.838 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.944.851 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.884 [plugin_manager.cc:439]619431 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619431,python3.9):2025-04-03-09:35:43.944.901 [ge_executor.cc:259]619431 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.251 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.285 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.304 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.400 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.416 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.489 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.502 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.945.533 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.577 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.592 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.945.619 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.663 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.677 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.740 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.754 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.816 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.830 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.945.857 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.907 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.920 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.945.948 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.945.990 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.003 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.067 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.080 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.146 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.159 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.185 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.224 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.237 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.264 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.302 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.315 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.376 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.391 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.460 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.474 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.500 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.539 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.552 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.578 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.618 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.631 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.691 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.703 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.763 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.776 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.801 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.838 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.852 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.879 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.918 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.946.937 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.946.962 [opsproto_manager.cc:123]619431 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.001 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.062 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.076 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.088 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.099 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.947.110 [opsproto_manager.cc:147]619431 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.428 [operator_factory_impl.cc:215]619431 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.454 [operator_factory_impl.cc:269]619431 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619431,python3.9):2025-04-03-09:35:43.947.483 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.952.715 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.426 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.529 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.575 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.607 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.639 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.666 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.696 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.736 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.953.764 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:43.971.024 [operator_factory_impl.cc:368]619431 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619431,python3.9):2025-04-03-09:35:43.971.235 [operator_factory_impl.cc:284]619431 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.399 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.422 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.437 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.449 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.460 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619431,python3.9):2025-04-03-09:35:43.971.478 [operator_factory_impl.cc:254]619431 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619431,python3.9):2025-04-03-09:35:43.971.504 [operator_factory_impl.cc:340]619431 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619431,python3.9):2025-04-03-09:35:43.971.528 [operator_factory_impl.cc:307]619431 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.556 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.575 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.596 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.971.696 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.475 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.508 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.526 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.544 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.562 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.577 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.596 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.645 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.662 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.679 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.695 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.719 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.734 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.761 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.794 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.815 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.880 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.919 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.936 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.952 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.972.980 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.973.000 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.091 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.116 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.136 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.152 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.166 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.181 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.203 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.238 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.251 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.267 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.354 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.378 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.394 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.406 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.418 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.430 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.722 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.740 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.753 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.767 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.779 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.791 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.808 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.830 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.845 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.858 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.870 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.894 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.974.907 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.117 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.138 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.198 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.214 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.228 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.240 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.975.253 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.099 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.125 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.141 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.180 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.196 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.209 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.227 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.339 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.425 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.770 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.791 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.803 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.815 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.826 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.976.839 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.270 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.567 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.589 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.603 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.620 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.761 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.977.904 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.178 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.202 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.242 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.269 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.805 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.846 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.865 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.885 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.903 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.919 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.940 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.978.962 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.008 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.046 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.416 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.589 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.840 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619431,python3.9):2025-04-03-09:35:43.979.868 [operator_factory_impl.cc:222]619431 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.449 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.510 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.558 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.606 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.627 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.669 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.703 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.747 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.768 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.809 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.830 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.848 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.870 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.890 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.907 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.926 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.960 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.980.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.004 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.045 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.070 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.094 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.115 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.154 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.181 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.202 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.249 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.269 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.289 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.309 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.331 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.351 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.373 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.402 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.425 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.467 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.528 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.549 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.566 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.608 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.632 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.654 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.675 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.695 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.717 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.738 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.764 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.787 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.830 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.852 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.938 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.959 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.981.979 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.002 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.045 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.065 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.086 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.107 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.150 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.171 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.191 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.209 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.230 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.247 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.267 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.298 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.322 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.342 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.363 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.383 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.403 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.425 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.469 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.491 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.512 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.552 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.572 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.594 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.615 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.634 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.661 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.681 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.701 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.721 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.762 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.782 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.803 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.828 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.849 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.937 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.957 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.979 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.982.999 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.019 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.039 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.064 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.081 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.121 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.143 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.159 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.176 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.193 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.210 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.232 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.293 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.314 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.339 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.357 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.377 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.394 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.414 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.431 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.447 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.464 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.481 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.499 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.606 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.631 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.679 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.701 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.722 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.744 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.764 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.785 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.805 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.826 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.846 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.869 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.889 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.909 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.937 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.960 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.983.984 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.005 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.050 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.070 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.091 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.112 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.154 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.186 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.208 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.231 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.295 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.316 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.337 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.360 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.380 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.401 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.421 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.444 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.465 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.501 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.522 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.545 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.566 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.589 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.612 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.636 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.657 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.743 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.789 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.814 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.833 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.853 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.875 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.897 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.920 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.944 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.966 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.984.988 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.012 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.031 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.080 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.101 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.148 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.176 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.215 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.260 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.392 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.496 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.594 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.804 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.829 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.858 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.880 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.985.987 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.084 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.180 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.290 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.396 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.620 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.644 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.668 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.690 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.777 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.882 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.906 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.928 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.950 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.968 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.986.992 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.114 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.321 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.422 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.549 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.645 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.669 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.687 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.717 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.812 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.987.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.012 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.107 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.131 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.340 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.364 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.458 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.554 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.769 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.864 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.988.960 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.147 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.248 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.341 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.367 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.393 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.414 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.432 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.463 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.488 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.509 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.556 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.580 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.602 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.692 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.786 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.825 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.861 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.989.975 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.067 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.160 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.181 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.375 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.401 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.419 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.527 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.625 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.729 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.823 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.990.924 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.030 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.224 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.411 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.513 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.536 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.555 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.751 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.775 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.867 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.991.963 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.056 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.152 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.264 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.289 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.397 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.502 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.598 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.690 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.714 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.900 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.992.997 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.091 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.184 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.306 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.329 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.428 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.456 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.474 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.492 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.509 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.526 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.542 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.559 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.575 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.610 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.640 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.656 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.673 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.820 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.993.920 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.022 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.115 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.206 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.312 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.439 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.465 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.579 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.605 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.629 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.651 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.746 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.769 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.790 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.818 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.994.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.015 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.110 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.210 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.393 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.609 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.706 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.811 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.995.905 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.004 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.102 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.131 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.153 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.199 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.388 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.589 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.684 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.782 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.888 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.996.987 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.091 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.296 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.394 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.493 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.522 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.618 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.728 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.754 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.775 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.802 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.822 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.860 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.997.972 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.001 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.101 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.290 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.386 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.599 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.694 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.811 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.998.910 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.002 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.102 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.203 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.299 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.404 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.430 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.452 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.498 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.521 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.542 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.564 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.587 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.608 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.631 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.676 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.696 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.716 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.756 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.777 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.825 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:43.999.920 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.023 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.049 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.149 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.263 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.363 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.534 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.637 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.737 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.808 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.000.909 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.005 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.101 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.193 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.297 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.398 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.501 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.636 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.728 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.827 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.001.927 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.022 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.121 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.144 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.236 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.329 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.523 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.616 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.718 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.823 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.002.920 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.021 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.119 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.311 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.406 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.427 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.522 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.622 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.003.940 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.037 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.252 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.543 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.640 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.662 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.760 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.860 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.004.956 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.049 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.150 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.250 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.407 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.516 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.612 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.705 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.808 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.005.910 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.119 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.308 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.332 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.623 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.644 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.663 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.680 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.717 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.734 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.751 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.857 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.878 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.895 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.006.985 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.177 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.279 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.375 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.395 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.502 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.593 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.613 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.798 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.893 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.007.994 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.032 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.052 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.145 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.181 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.202 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.298 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.337 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.431 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.451 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.478 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.496 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.585 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.676 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.772 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.792 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.814 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.909 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.008.932 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.030 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.219 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.259 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.349 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.388 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.480 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.580 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.675 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.772 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.868 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.009.960 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.061 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.161 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.359 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.457 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.551 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.645 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.744 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.844 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.942 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.964 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.010.983 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.078 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.207 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.304 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.398 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.495 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.690 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.791 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.886 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.011.995 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.093 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.115 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.133 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.151 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.395 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.495 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.595 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.685 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.874 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.012.973 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.068 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.162 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.351 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.453 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.550 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.648 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.746 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.013.941 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.049 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.145 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.235 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.332 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.526 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.618 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.718 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.837 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.014.934 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.031 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.334 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.536 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.635 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.737 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.829 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.015.927 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.127 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.233 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.344 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.623 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.824 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.016.926 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.020 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.115 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.415 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.606 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.791 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.017.904 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.042 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.076 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.106 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.137 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.163 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.191 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.217 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.251 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.275 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.293 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.311 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.328 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.347 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.375 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.396 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.490 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.591 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.686 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.780 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.874 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.018.978 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.076 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.157 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.189 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.377 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.476 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.579 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.674 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.776 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.820 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.838 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.855 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.019.980 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.010 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.038 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.068 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.094 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.127 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.167 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.196 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.246 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.275 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.299 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.351 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.386 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.420 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.467 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.535 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.637 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.665 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.864 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.020.964 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.153 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.277 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.372 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.470 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.656 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.864 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.021.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.054 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.134 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.308 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.407 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.504 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.605 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.772 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.857 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.022.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.049 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.144 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.327 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.458 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.553 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.742 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.023.939 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.039 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.184 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.394 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.491 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.775 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.799 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.822 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.872 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.024.966 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.135 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.164 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.187 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.376 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.473 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.566 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.658 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.755 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.778 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.931 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.025.953 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.022 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.217 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.314 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.510 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.606 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.711 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.810 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.884 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.026.913 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.005 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.103 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.203 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.300 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.396 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.489 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.588 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.668 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.692 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.027.905 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.243 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.452 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.477 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.499 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.521 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.542 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.584 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.621 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.638 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.658 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.028.817 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.007 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.111 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.141 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.164 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.255 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.354 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.462 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.557 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.664 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.687 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.707 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.724 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.743 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.836 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.029.930 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.034 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.137 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.163 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.185 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.289 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.314 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.510 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.792 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.030.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.045 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.070 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.093 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.166 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.191 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.213 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.235 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.280 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.304 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.326 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.350 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.372 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.468 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.490 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.617 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.644 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.670 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.729 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.792 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.814 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.031.836 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.032.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.009 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.090 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.165 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.245 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.314 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.424 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.517 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.617 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.688 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.763 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.834 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.857 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.033.986 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.179 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.265 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.341 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.415 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.736 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.759 [opsproto_manager.cc:162]619431 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.806 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.034.847 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.350 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.406 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.427 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.507 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.521 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.582 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.596 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.608 [opp_so_manager.cc:56]619431 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.620 [plugin_manager.cc:540]619431 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.635 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.657 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.672 [plugin_manager.cc:547]619431 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.035.684 [plugin_manager.cc:206]619431 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.040 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.081 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.096 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.110 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.154 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.490 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.521 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.534 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.546 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.579 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.616 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.631 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.644 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.657 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.687 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.699 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.036.710 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.036.722 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.046 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.079 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.092 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.104 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.137 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.448 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.480 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.493 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.505 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.537 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.569 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.582 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.593 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.605 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.634 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.648 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.037.659 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.683 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.037.996 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.031 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.044 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.056 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.088 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.399 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.431 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.445 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.458 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.492 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.526 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.539 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.552 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.564 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.592 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.606 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.618 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.643 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.038.952 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.986 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.038.998 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.011 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.043 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.357 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.388 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.400 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.413 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.445 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.477 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.490 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.501 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.513 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.542 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.556 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.587 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.601 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.039.917 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.949 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.961 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.039.974 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.016 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.348 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.386 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.399 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.410 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.443 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.483 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.497 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.507 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.517 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.552 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.565 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.040.589 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.601 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.611 [plugin_manager.cc:218]619431 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.629 [plugin_manager.cc:422]619431 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.640 [plugin_manager.cc:155]619431 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.661 [plugin_manager.cc:158]619431 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.672 [plugin_manager.cc:426]619431 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.040.982 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.041.015 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.041.027 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.041.040 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.072 [plugin_manager.cc:439]619431 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.368 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.404 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.423 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.528 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.546 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.041.558 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.043.787 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.043.827 [op_impl_registry_holder_manager.cc:216]619431 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.043.844 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.043.888 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.044.389 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.044.511 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.044.528 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.044.540 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.046.618 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.046.653 [op_impl_registry_holder_manager.cc:216]619431 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.046.669 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.046.705 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.032 [op_impl_space_registry.cc:80]619431 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.064 [op_impl_space_registry.cc:83]619431 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.084 [op_impl_space_registry.cc:80]619431 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.097 [op_impl_space_registry.cc:83]619431 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619431,python3.9):2025-04-03-09:35:44.047.112 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.160 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.047.195 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.047.281 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.047.297 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.047.307 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.035 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.048.059 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.146 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.173 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.186 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.805 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.048.828 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.048.865 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.048.910 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.048.990 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.005 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.018 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.610 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.049.631 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.722 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.737 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.049.747 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.050.324 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.050.348 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.050.385 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.050.417 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.050.495 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.050.510 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.050.521 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.091 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.051.127 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.212 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.226 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.241 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.796 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.051.817 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.051.858 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.051.888 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.971 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.985 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.051.998 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.052.576 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.052.600 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619431,python3.9):2025-04-03-09:35:44.052.685 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.052.699 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.052.711 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.053.266 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.053.298 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.053.335 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.053.365 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.053.401 [opp_so_manager.cc:97]619431 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.053.469 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.053.509 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.053.524 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.053.533 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.586 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.622 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.642 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.658 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.674 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.689 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.721 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.735 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.765 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.793 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.812 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.828 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.859 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.897 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.932 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.965 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.083.996 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.012 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.047 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.065 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.080 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.096 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.111 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.126 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.175 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.230 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.245 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.267 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.300 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.317 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.333 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.364 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.379 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.395 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.411 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.426 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.461 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.478 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.494 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.526 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.543 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.559 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.573 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.591 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.608 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.623 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.638 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.653 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.672 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.686 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.707 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.724 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.755 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.784 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.799 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.813 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.827 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.844 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.859 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.874 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.889 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.903 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.918 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.933 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.948 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.965 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.084.998 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.045 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.059 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.075 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.090 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.108 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.155 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.170 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.186 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.201 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.221 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.237 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.252 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.269 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.301 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.316 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.331 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.346 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.362 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.377 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.393 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.410 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.428 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.449 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.466 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.483 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.501 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.517 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.538 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.564 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.582 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.598 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.616 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.633 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.650 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.668 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.686 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.703 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.740 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.758 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.814 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.834 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.851 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.868 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.888 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.908 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.927 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.972 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.085.989 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.007 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.114 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.159 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.177 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.194 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.216 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.233 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.251 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.268 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.287 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.303 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.322 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.338 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.358 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.376 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.392 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.410 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.446 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.467 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.486 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.504 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.523 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.545 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.565 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.584 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.620 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.642 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.660 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.697 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.714 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.730 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.747 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.766 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.782 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.802 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.819 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.836 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.853 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.869 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.886 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.902 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.922 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.939 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.975 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.086.991 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.011 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.030 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.046 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.063 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.083 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.100 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.160 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.176 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.197 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.213 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.233 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.269 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.286 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.303 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.336 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.353 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.388 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.405 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.422 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.457 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.474 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.578 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.679 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.778 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.874 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.087.978 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.013 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.035 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.053 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.168 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.275 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.373 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.470 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.572 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.681 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.784 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.828 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.088.941 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.040 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.063 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.102 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.163 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.259 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.357 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.471 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.571 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.799 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.823 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.841 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.858 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.089.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.053 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.158 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.282 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.376 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.496 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.895 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.090.917 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.008 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.104 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.204 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.306 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.414 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.539 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.567 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.586 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.607 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.625 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.642 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.661 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.696 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.714 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.733 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.821 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.922 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.091.982 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.002 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.019 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.244 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.342 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.365 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.464 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.585 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.610 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.710 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.820 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.092.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.026 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.325 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.427 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.525 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.624 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.737 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.763 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.783 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.879 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.978 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.093.998 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.096 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.191 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.290 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.392 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.492 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.515 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.618 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.718 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.819 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.919 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.094.942 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.057 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.151 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.256 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.355 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.457 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.561 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.605 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.784 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.820 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.836 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.859 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.876 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.893 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.910 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.929 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.963 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.980 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.095.998 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.098 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.218 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.421 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.528 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.623 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.882 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.905 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.923 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.096.945 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.041 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.082 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.100 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.194 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.402 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.506 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.606 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.702 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.805 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.097.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.032 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.134 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.231 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.330 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.431 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.455 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.477 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.500 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.598 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.899 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.098.996 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.102 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.207 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.308 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.411 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.625 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.728 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.827 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.849 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.099.947 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.046 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.069 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.089 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.136 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.164 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.189 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.209 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.305 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.334 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.437 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.533 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.630 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.736 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.837 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.100.940 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.039 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.157 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.256 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.360 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.457 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.567 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.666 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.767 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.789 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.808 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.826 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.858 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.878 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.898 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.936 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.954 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.971 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.101.991 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.008 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.027 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.044 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.081 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.097 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.121 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.320 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.343 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.450 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.550 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.817 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.102.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.018 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.089 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.292 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.394 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.501 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.701 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.903 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.103.926 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.026 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.242 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.342 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.467 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.576 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.672 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.776 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.874 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.104.973 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.072 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.177 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.277 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.384 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.594 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.805 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.830 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.105.934 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.034 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.136 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.246 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.448 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.551 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.752 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.850 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.106.950 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.059 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.080 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.176 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.283 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.386 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.483 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.586 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.890 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.107.991 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.218 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.327 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.433 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.537 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.764 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.868 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.108.970 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.077 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.118 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.138 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.156 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.174 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.193 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.234 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.328 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.349 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.368 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.464 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.565 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.672 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.785 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.884 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.904 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.109.922 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.149 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.354 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.452 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.549 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.572 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.610 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.711 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.734 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.755 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.853 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.875 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.110.994 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.032 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.050 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.154 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.260 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.372 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.395 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.413 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.517 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.538 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.640 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.738 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.866 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.885 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.111.980 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.005 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.026 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.245 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.359 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.463 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.566 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.668 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.112.984 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.083 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.188 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.285 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.400 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.509 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.620 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.770 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.867 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.889 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.113.909 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.010 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.219 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.320 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.421 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.630 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.739 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.943 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.968 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.114.992 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.012 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.109 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.211 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.321 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.419 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.638 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.740 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.850 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.115.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.167 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.278 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.383 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.591 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.690 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.794 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.116.898 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.003 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.111 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.212 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.313 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.410 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.516 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.615 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.721 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.117.947 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.166 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.274 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.376 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.474 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.574 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.677 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.780 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.885 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.118.987 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.086 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.188 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.400 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.498 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.804 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.119.912 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.020 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.124 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.240 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.343 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.446 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.556 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.684 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.712 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.734 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.761 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.782 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.802 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.823 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.865 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.883 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.906 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.944 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.964 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.120.984 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.083 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.192 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.290 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.389 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.497 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.789 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.815 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.121.912 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.024 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.130 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.230 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.334 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.462 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.480 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.498 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.518 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.678 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.722 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.745 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.766 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.787 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.808 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.830 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.852 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.871 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.941 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.963 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.122.986 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.018 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.041 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.115 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.220 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.240 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.340 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.538 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.747 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.849 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.870 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.123.968 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.069 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.193 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.315 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.420 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.522 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.621 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.797 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.875 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.124.984 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.084 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.208 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.316 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.417 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.499 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.667 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.768 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.872 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.125.970 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.051 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.156 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.180 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.285 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.384 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.495 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.600 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.812 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.126.919 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.020 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.123 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.227 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.325 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.471 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.492 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.525 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.551 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.595 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.897 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.127.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.013 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.239 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.453 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.558 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.580 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.752 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.776 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.856 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.128.959 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.160 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.259 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.362 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.475 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.585 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.688 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.763 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.785 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.884 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.129.987 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.200 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.300 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.399 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.502 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.579 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.602 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.709 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.811 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.130.908 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.008 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.110 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.152 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.170 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.188 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.209 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.228 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.247 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.264 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.293 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.315 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.332 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.351 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.368 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.472 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.570 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.673 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.696 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.720 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.824 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.131.930 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.032 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.139 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.256 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.285 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.307 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.330 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.353 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.448 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.550 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.757 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.800 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.906 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.132.937 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.038 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.141 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.247 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.350 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.461 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.486 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.584 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.609 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.713 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.736 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.757 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.777 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.793 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.812 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.829 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.849 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.866 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.889 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.907 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.923 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.944 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.133.979 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.081 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.114 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.133 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.234 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.256 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.294 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.312 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.330 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.348 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.369 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.385 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.403 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.528 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.727 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.799 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.875 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.134.952 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.280 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.354 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.431 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.588 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.731 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.859 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.135.965 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.065 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.223 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.291 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.312 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.136.572 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.145.176 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.149.570 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.172.149 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.172.505 [plugin_manager.cc:554]619431 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619431,python3.9):2025-04-03-09:35:44.172.526 [plugin_manager.cc:134]619431 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.172.571 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.172.586 [plugin_manager.cc:145]619431 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.172.598 [plugin_manager.cc:557]619431 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.172.611 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.172.636 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.139 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.192 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.214 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.288 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.303 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.364 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.394 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.407 [opp_so_manager.cc:66]619431 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.422 [plugin_manager.cc:616]619431 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.431 [plugin_manager.cc:108]619431 GetOppPath:Enter get opp path schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.456 [plugin_manager.cc:117]619431 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.473 [plugin_manager.cc:624]619431 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.487 [plugin_manager.cc:206]619431 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.833 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.173.878 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.173.894 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.173.911 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.173.954 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.285 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.319 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.332 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.346 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.378 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.422 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.451 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.463 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.478 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.495 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.824 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.859 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.873 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.174.887 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.174.920 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.254 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.289 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.303 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.316 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.351 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.392 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.406 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.417 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.431 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.458 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.781 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.816 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.832 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.175.844 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.175.875 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.223 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.263 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.278 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.292 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.329 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.370 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.385 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.397 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.408 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.423 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.751 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.800 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.815 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.176.828 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.176.858 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.194 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.231 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.245 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.256 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.290 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.330 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.345 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.358 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.371 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.385 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.705 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.738 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.751 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.177.765 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.177.798 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.136 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.169 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.184 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.198 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.230 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.264 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.277 [plugin_manager.cc:1026]619431 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.291 [plugin_manager.cc:322]619431 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.303 [plugin_manager.cc:212]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.316 [plugin_manager.cc:215]619431 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.329 [plugin_manager.cc:218]619431 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.348 [plugin_manager.cc:422]619431 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.361 [plugin_manager.cc:155]619431 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.386 [plugin_manager.cc:158]619431 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.399 [plugin_manager.cc:426]619431 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.725 [plugin_manager.cc:250]619431 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.758 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.772 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.178.785 [plugin_manager.cc:1063]619431 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.178.819 [plugin_manager.cc:439]619431 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.118 [plugin_manager.cc:996]619431 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.146 [plugin_manager.cc:1011]619431 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.166 [plugin_manager.cc:1015]619431 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.179.224 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.321 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.335 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.179.347 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.180.721 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff3821c2c0, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.180.753 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.180.774 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.180.791 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.180.823 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.113 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff38220978, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.136 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.155 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.174 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.188 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.447 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.181.478 [op_impl_registry_holder_manager.cc:216]619431 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.494 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.532 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.782 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.181.837 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.941 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.957 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.181.969 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.278 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.316 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.333 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.358 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.373 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.644 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff381cd9b0, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.664 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.680 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.694 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.711 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.183.982 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff381d0da0, registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.007 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.021 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.034 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.047 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.298 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.327 [op_impl_registry_holder_manager.cc:216]619431 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.341 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.374 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.666 [op_impl_space_registry.cc:84]619431 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.684 [op_impl_space_registry.cc:85]619431 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.710 [op_impl_space_registry.cc:86]619431 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.721 [op_impl_space_registry.cc:87]619431 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.731 [op_impl_space_registry.cc:93]619431 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.748 [op_impl_space_registry.cc:84]619431 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.761 [op_impl_space_registry.cc:85]619431 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.772 [op_impl_space_registry.cc:86]619431 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.782 [op_impl_space_registry.cc:87]619431 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.792 [op_impl_space_registry.cc:93]619431 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619431,python3.9):2025-04-03-09:35:44.184.806 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.857 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.893 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.184.930 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.025 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.039 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.051 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.593 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.185.616 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.185.662 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.768 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.786 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.185.796 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.186.415 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.186.439 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.186.488 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.186.526 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.186.564 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.186.654 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.186.668 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.186.679 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.187.134 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.187.154 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.187.198 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.187.291 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.187.314 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.187.328 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.187.887 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.187.908 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.187.953 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.187.993 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.188.027 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.120 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.135 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.145 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.600 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.188.622 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.188.666 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.758 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.773 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.188.795 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.189.362 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.189.383 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.189.429 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.189.470 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.189.511 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.189.616 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.189.632 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.189.644 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.190.079 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.097 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.143 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.190.236 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.190.252 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.190.263 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.190.808 [op_impl_space_registry.cc:240]619431 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.836 [opp_so_manager.cc:48]619431 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.880 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.912 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.946 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.190.969 [opp_so_manager.cc:126]619431 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.191.000 [opp_so_manager.cc:123]619431 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.191.089 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.191.104 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.191.150 [plugin_manager.cc:1126]619431 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.191.167 [plugin_manager.cc:1130]619431 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.191.182 [opp_so_manager.cc:45]619431 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.204.269 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.205.030 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.205.077 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.205.105 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.207.774 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.224.194 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.224.242 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.224.260 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.224.295 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.224.321 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.224.343 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.224.363 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.228.183 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.228.216 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.228.231 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.228.246 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.228.265 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.228.283 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.228.300 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.322.877 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.365 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.447 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.524 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.600 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.669 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.747 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.768 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.833 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.853 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.323.892 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.019 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.095 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.175 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.250 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.317 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.386 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.406 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.472 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.493 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.512 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.723 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.794 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.863 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.324.929 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.002 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.072 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.146 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.214 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.351 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.416 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.483 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.554 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.627 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.325.711 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.331.493 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.331.619 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.331.733 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.153 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.274 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.381 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.545 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.655 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.808 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.332.951 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.061 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.188 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.324 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.439 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.555 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.578 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.735 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.853 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.333.996 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.111 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.229 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.373 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.494 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.773 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.334.887 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.004 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.138 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.248 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.368 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.479 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.592 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.711 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.739 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.335.959 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.080 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.236 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.467 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffef290d360, registered count:5 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.496 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.515 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.532 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.549 [op_check.cc:71]619431 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.567 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.693 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.336.810 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.008 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.190 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.304 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.326 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.430 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.453 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.560 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.677 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.810 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.337.919 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.027 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.355 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.379 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.489 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.598 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.338.922 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.031 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.141 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.264 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.469 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.339.937 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.067 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.198 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.323 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.433 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.550 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.829 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.340.996 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.018 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffef2997e08, registered count:7 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.183 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.201 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffef29a0298, registered count:7 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.317 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.429 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.584 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.807 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.341.926 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.033 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.259 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.365 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.391 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.504 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.620 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.729 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.843 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.342.968 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.088 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.201 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.315 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.462 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.511 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.691 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.805 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.343.916 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.032 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.140 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.274 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.296 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.320 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.432 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.539 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.646 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.840 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.344.950 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.062 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.184 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.296 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.407 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.528 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.674 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.779 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.345.802 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.092 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.202 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.312 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.425 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.651 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.758 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.346.870 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.347.075 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.347.183 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.347.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.349.834 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.349.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.349.894 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.025 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.153 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.261 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.382 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.491 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.604 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.715 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.831 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.964 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.350.987 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.097 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.229 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.357 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.474 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.589 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.714 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.351.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.352.258 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.352.414 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.352.526 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.352.644 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.352.761 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.353.441 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.354.277 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.354.620 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.355.093 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.355.716 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.355.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.355.977 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.356.128 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.356.250 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.356.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.358.839 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.359.716 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.359.888 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.360.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.361.117 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.361.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.361.439 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.361.925 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.165 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.302 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.412 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.532 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.643 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.756 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.876 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.362.992 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.108 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.385 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.821 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.363.934 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.236 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.418 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.535 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.676 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.869 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.364.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.090 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.266 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.380 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.548 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.664 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.900 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.365.922 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.031 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.055 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.172 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.282 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.304 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.409 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.523 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.637 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.829 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.366.944 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.367.151 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.367.311 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.367.492 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.367.662 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.367.845 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.044 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.171 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.285 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.401 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.515 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.734 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.847 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.368.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.071 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.182 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.306 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.424 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.450 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.573 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.369.805 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.370.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.373.601 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.373.826 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.374.037 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.374.391 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.374.574 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.374.725 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.375.571 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.375.746 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.375.886 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.217 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.355 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.524 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.680 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.832 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.376.961 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.075 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.173 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.497 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.524 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.544 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.563 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.588 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.609 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.628 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.646 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.663 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.682 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.700 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.729 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.766 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.784 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.801 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.822 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.860 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.878 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.896 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.914 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.934 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.953 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.969 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.377.986 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.009 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.026 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.069 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.087 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.105 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.122 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.139 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.158 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.178 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.202 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.219 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.237 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.271 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.291 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.308 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.327 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.344 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.360 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.378 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.398 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.419 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.437 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.453 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.472 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.488 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.507 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.529 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.546 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.695 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.378.844 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.010 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.034 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.053 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.071 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.099 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.243 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.361 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.474 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.689 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.800 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.379.956 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.080 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.369 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.494 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.519 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.540 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.652 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.677 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.699 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.814 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.380.977 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.132 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.279 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.391 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.593 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.706 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.381.890 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.014 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.139 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.163 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.182 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.331 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.483 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.631 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.783 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.382.970 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.123 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.238 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.397 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.555 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.733 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.861 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.383.973 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.085 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.215 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.335 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.641 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.873 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.384.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.385.103 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.385.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.385.682 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.125 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.369 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.394 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.505 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.622 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.739 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.863 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.386.977 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.097 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.205 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.316 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.481 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.607 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.725 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.387.915 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.029 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.150 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.283 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.471 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.658 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.777 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.388.904 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.027 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.142 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.254 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.489 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.759 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.389.920 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.390.199 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.390.390 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.390.560 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.390.749 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.390.966 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.164 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.382 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.405 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.423 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.440 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.464 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.484 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.503 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.391.522 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.392.319 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.392.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.393.154 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.393.358 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.393.560 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.393.985 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.394.105 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.394.271 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.396.834 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.398.717 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.398.842 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.398.962 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.399.076 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.399.370 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.399.525 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.399.718 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.057 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.085 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.109 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.129 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.151 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.357 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.381 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.542 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.704 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.400.884 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.058 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.243 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.283 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.441 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.616 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.788 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.401.954 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.129 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.307 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.468 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.650 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.676 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.782 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.906 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.402.931 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.281 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.309 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.427 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.545 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.666 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.796 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.403.907 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.404.019 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.404.226 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.404.433 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.404.822 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.405.222 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.405.419 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.405.758 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.405.958 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.159 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.403 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.558 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.585 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.607 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.406.630 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.407.775 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.241 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.417 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.573 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.754 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.408.913 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.070 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.224 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.251 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.407 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.571 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.725 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.409.906 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.410.072 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.410.479 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.410.852 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.411.113 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.411.309 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.412.452 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.412.930 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.413.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.413.681 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.413.854 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.413.981 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.152 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.316 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.344 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.481 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.508 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.862 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.414.979 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.415.143 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.415.311 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.415.438 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.415.746 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.415.948 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.120 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.253 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.391 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.560 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.750 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.946 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.416.974 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.417.150 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.417.174 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.417.333 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.417.538 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.419.023 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.424.870 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.424.917 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.427.512 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.427.792 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.428.090 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.428.445 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.428.659 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.428.871 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.048 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.206 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.399 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.542 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.642 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.429.910 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.430.028 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.430.209 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.430.712 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.430.885 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.052 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.079 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.195 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.318 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.505 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.698 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.431.904 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.068 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.210 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.333 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.361 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.583 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.793 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.432.956 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.072 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.246 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.273 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.416 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.535 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.559 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.579 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.726 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.433.940 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.041 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.284 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.487 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.515 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.543 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.741 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.434.773 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.437.894 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.437.988 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.012 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.084 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.179 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.199 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.266 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.442 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.461 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.582 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.601 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.618 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.633 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.797 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.438.816 [tuning_tiling_registry.cc:30]619431 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.440.032 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffef32dc0b0, registered count:7 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.440.070 [op_check.cc:24]619431 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffef32dc310, registered count:5 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.440.504 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.441.056 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.442.800 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.442.976 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.443.059 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.443.454 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.443.918 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.474.625 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.474.760 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.474.955 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.475.037 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.475.184 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.475.333 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.028 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.116 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.404 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.488 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.647 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.476.893 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.477.070 [op_impl_registry.cc:122]619431 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619431,python3.9):2025-04-03-09:35:44.477.788 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.483.376 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619431,python3.9):2025-04-03-09:35:44.485.520 [op_impl_registry_holder_manager.cc:208]619431 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.564.220 [op_impl_space_registry.cc:109]619431 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.565.533 [op_impl_space_registry.cc:84]619431 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.565.556 [op_impl_space_registry.cc:85]619431 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.565.582 [op_impl_space_registry.cc:86]619431 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.565.594 [op_impl_space_registry.cc:87]619431 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.565.606 [op_impl_space_registry.cc:93]619431 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619431,python3.9):2025-04-03-09:35:44.566.623 [op_impl_space_registry.cc:273]619431 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.566.908 [plugin_manager.cc:564]619431 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619431,python3.9):2025-04-03-09:35:44.566.929 [plugin_manager.cc:134]619431 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.566.982 [file_utils.cc:47]619431 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.566.997 [plugin_manager.cc:145]619431 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619431,python3.9):2025-04-03-09:35:44.567.011 [plugin_manager.cc:567]619431 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.077 [mem_manager.cc:31]619431 Initialize:MemManager Initialize. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.109 [rts_caching_mem_allocator.cc:131]619431 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.125 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.135 [graph_mem_allocator.cc:17]619431 Initialize:MemoryAllocator::Initialize. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.147 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.158 [graph_mem_allocator.cc:17]619431 Initialize:MemoryAllocator::Initialize. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.169 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.183 [caching_allocator.cc:391]619431 FreeBlocks:Free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.195 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.208 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.217 [caching_allocator.cc:391]619431 FreeBlocks:Free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.227 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.239 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.250 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.264 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.277 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.288 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.298 [session_scope_mem_allocator.cc:73]619431 FreeAllMemory:Free all memory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.322 [mem_manager.cc:102]619431 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.333 [session_scope_mem_allocator.cc:73]619431 FreeAllMemory:Free all memory [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.377 [dump_manager.cc:152]619431 Init:exception dump is not enabled [INFO] GE(619431,python3.9):2025-04-03-09:35:44.567.418 [ge_executor.cc:315]619431 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.436 [acl.cpp:321]619431 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.451 [dump.cpp:841]619431 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.590 [dump.cpp:759]619431 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.606 [dump.cpp:594]619431 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.619 [dump.cpp:224]619431 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.657 [dump.cpp:703]619431 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.673 [dump.cpp:771]619431 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.567.684 [dump.cpp:703]619431 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.567.744 [adx_datadump_server.cpp:77][tid:619431] start to do dump init [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.567.762 [adx_dump_record.cpp:210][tid:619431] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.567.774 [adx_dump_record.cpp:214][tid:619431] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.567.994 [adx_dump_record.cpp:627][tid:619613] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.001 [adx_server_manager.cpp:149][tid:619612] server manager add component (10) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.038 [adx_server_manager.cpp:165][tid:619612] server manager components init successfully [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.184 [adx_server_manager.cpp:203][tid:619614] Run Server(0) Process [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.240 [adx_server_manager.cpp:369][tid:619614] device up 0 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.256 [hdc_comm_opt.cpp:59][tid:619614] open device[0] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.444 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.461 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.476 [hdc_api.cpp:119][tid:619614] logDevId 0 create HDC server successfully [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.517 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.533 [adx_server_manager.cpp:369][tid:619614] device up 1 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.547 [hdc_comm_opt.cpp:59][tid:619614] open device[1] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.679 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.700 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.713 [hdc_api.cpp:119][tid:619614] logDevId 1 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.724 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.738 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.749 [adx_server_manager.cpp:369][tid:619614] device up 2 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.761 [hdc_comm_opt.cpp:59][tid:619614] open device[2] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.898 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.568.911 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.922 [hdc_api.cpp:119][tid:619614] logDevId 2 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.933 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.945 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.956 [adx_server_manager.cpp:369][tid:619614] device up 3 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.568.965 [hdc_comm_opt.cpp:59][tid:619614] open device[3] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.094 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.108 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.121 [hdc_api.cpp:119][tid:619614] logDevId 3 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.131 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.142 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.152 [adx_server_manager.cpp:369][tid:619614] device up 4 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.163 [hdc_comm_opt.cpp:59][tid:619614] open device[4] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.300 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.313 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.324 [hdc_api.cpp:119][tid:619614] logDevId 4 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.333 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.344 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.354 [adx_server_manager.cpp:369][tid:619614] device up 5 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.365 [hdc_comm_opt.cpp:59][tid:619614] open device[5] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.438 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.458 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.472 [hdc_api.cpp:119][tid:619614] logDevId 5 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.485 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.497 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.507 [adx_server_manager.cpp:369][tid:619614] device up 6 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.518 [hdc_comm_opt.cpp:59][tid:619614] open device[6] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.653 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.667 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.679 [hdc_api.cpp:119][tid:619614] logDevId 6 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.692 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.704 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.712 [adx_server_manager.cpp:369][tid:619614] device up 7 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.721 [hdc_comm_opt.cpp:59][tid:619614] open device[7] server[13] [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.794 [drv_log_user.c:623][ascend][curpid:619431,619614][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.569.807 [hdc_server.c:363][ascend][curpid:619431,619614][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.819 [hdc_api.cpp:119][tid:619614] logDevId 7 create HDC server successfully [WARNING] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.828 [adx_hdc_epoll.cpp:34][tid:619614] hdc epoll has been created. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.569.841 [adx_server_manager.cpp:111][tid:619614] create server info [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.571.088 [adx_server_manager.cpp:457][tid:619431] The server is initialized after waiting 3 times. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.571.109 [ge_executor.cc:1349]619431 SetDump:Start to set dump config [INFO] GE(619431,python3.9):2025-04-03-09:35:44.571.125 [dump_manager.cc:201]619431 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.571.161 [dump_manager.cc:261]619431 SetExceptionDump:Enable brief exception dump. [INFO] GE(619431,python3.9):2025-04-03-09:35:44.571.194 [ge_executor.cc:1355]619431 SetDump:Set dump config successfully [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.571.225 [dump_manager.cpp:146][tid:619431] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:44.572.222 [dump_manager.cpp:301][tid:619431] Register exception callback, registered: 1 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.243 [dump.cpp:825]619431 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.262 [acl.cpp:327]619431 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.274 [acl.cpp:329]619431 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.288 [acl_resource_manager.cpp:248]619431 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.398 [acl_resource_manager.cpp:275]619431 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.416 [acl_resource_manager.cpp:278]619431 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.427 [acl.cpp:335]619431 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.438 [acl.cpp:336]619431 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.499 [json_parser.cpp:260]619431 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.515 [acl.cpp:343]619431 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:44.572.533 [prof_atls_plugin.cpp:210] (tid:619431) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.549 [acl.cpp:351]619431 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.561 [profiling.cpp:141]619431 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.575 [profiling.cpp:135]619431 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.637 [profiling.cpp:167]619431 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619431.json], errorCode = 0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.652 [profiling.cpp:172]619431 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619431.json] is empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.663 [profiling.cpp:176]619431 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.673 [profiling.cpp:179]619431 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.572.684 [profiling.cpp:92]619431 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:44.573.055 [ai_drv_dev_api.cpp:316] >>> (tid:619431) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:44.573.186 [msprof_callback_impl.cpp:132] >>> (tid:619431) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.200 [profiling.cpp:122]619431 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.213 [profiling.cpp:186]619431 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.223 [hash_utils.cpp:108]619431 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619431.json [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.259 [hash_utils.cpp:128]619431 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619431.json, hash: 3917491750093904922 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.273 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.303 [acl.cpp:280]619431 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.314 [acl.cpp:385]619431 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.573.333 [runtime.cc:5394] 619431 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.573.358 [api_impl.cc:7407] 619431 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.573.379 [api_impl.cc:4211] 619431 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.573.394 [acl.cpp:420]619431 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.573.566 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.573.661 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.573.716 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.573.735 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.578157] [info] [619431] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:44.578.779 [prof_atls_plugin.cpp:210] (tid:619431) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.579.488 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.579.519 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.579.594 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 3, rank size = 8. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.579.612 [device.cpp:143]619431 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.638 [api_impl.cc:2999] 619431 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.654 [device.cc:306] 619431 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.667 [runtime.cc:3393] 619431 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.683 [runtime.cc:3560] 619431 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.695 [api_impl.cc:2987] 619431 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.708 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.728 [api_impl.cc:2992] 619431 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.745 [api_impl.cc:3025] 619431 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.579.758 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.579.769 [acl.cpp:121]619431 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.579.789 [device.cpp:154]619431 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.579.852 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.579.875 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is _npu_log, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.896 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.908 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.579.920 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.579.957 [queue_client.c:151][ascend][curpid:619431,619431][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619431) [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.580.131 [queue_client.c:305][ascend][curpid:619431,619431][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=3; vfid=0; hostpid=619431; master_pid=619431; proc_type=0; devpid=31148). [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.584.153 [queue_client.c:362][ascend][curpid:619431,619431][drv][queuemng][QueueInit]QueueInit device. (devid=3; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.387 [npu_driver.cc:4114] 619431 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.404 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.418 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=_npu_log. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.584.699 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.584.714 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.584.850 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.584.879 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.584.910 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.922 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.584.931 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.936 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.949 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.961 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.584.972 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.022 [npu_driver.cc:4087] 619431 MemQueueInit: repeated init [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.036 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.055 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=ms_tensor_dump. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.585.253 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.268 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.374 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.404 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.585.405 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.426 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.427 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.441 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.446 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.456 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.468 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.503 [npu_driver.cc:4087] 619431 MemQueueInit: repeated init [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.515 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.526 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=ms_tensor_summary. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.585.693 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.708 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.891 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.919 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.585.923 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.585.941 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.947 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.958 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.961 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.983 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.585.993 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.026 [npu_driver.cc:4087] 619431 MemQueueInit: repeated init [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.039 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.050 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=ms_image_summary. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.586.214 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.229 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.327 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.357 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.586.360 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.377 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.381 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.394 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.397 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.407 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.416 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.449 [npu_driver.cc:4087] 619431 MemQueueInit: repeated init [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.461 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.472 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=ms_scalar_summary. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.586.632 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.646 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.748 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.586.772 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.778 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.586.801 [tensor_data_transfer.cpp:898]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.812 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.819 [api_impl.cc:6077] 619431 MemQueueInit: Start to init queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.824 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.829 [npu_driver.cc:4076] 619431 MemQueueInit: init mem queue, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.839 [npu_driver.cc:4026] 619431 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.872 [npu_driver.cc:4087] 619431 MemQueueInit: repeated init [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.884 [api_impl.cc:6040] 619431 MemQueueCreate: Start to create queue on devId 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.586.896 [npu_driver.cc:3931] 619431 MemQueueCreate: create mem queue, deviceId=3, name=ms_histogram_summary. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.587.062 [queue_interface.c:182][ascend][curpid:619431,619431][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.587.077 [tensor_data_transfer.cpp:940]619431 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 3, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.587.251 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.587.276 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.587.287 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.587.295 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.587.307 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.587.471 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.587.500 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.514 [plugin_manager.cc:25] [619431]hcom ops plugin init start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.541 [plugin_manager.cc:38] [619431]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.558 [plugin_manager.cc:42] [619431]hcom running normal mode. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.602 [hcom_plugin.cc:50] [619431]Initialize start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.614 [hcom_plugin.cc:52] [619431]initializeCount_ : 1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.656 [hcom_plugin.cc:60] [619431]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.684 [hcom_plugin.cc:63] [619431]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.697 [hcom_plugin.cc:329] [619431]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.721 [hcom_plugin.cc:281] [619431]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:44.587.740 [hcom_plugin.cc:217] [619431]option profiling mode is not found. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.757 [hcom_plugin.cc:164] [619431][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.769 [hcom_plugin.cc:180] [619431][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.781 [hcom_plugin.cc:244] [619431]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.930 [hcom_plugin.cc:82] [619431]hccl ops plugin init success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.944 [plugin_manager.cc:48] [619431]hcom ops plugin init end. ret[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.956 [hcom_plugin.cc:126] [619431]get hccl kernel info store start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.587.981 [hcom_plugin.cc:132] [619431]get hccl kernel info store finished. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.588.005 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.588.027 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.037 [hcom_executor.cc:29] [619431]Hcom Excutor Initialize start. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.088 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.104 [adapter_rts.cc:234] [619431][hrtGetDevice]deviceLogicId[3] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.137 [api_c_soc.cc:70] 619431 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.200 [hcom_common.cc:85] [619431][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.229 [hcom_common.cc:88] [619431][HcomGetCurHcomCtx] is no Used deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.253 [hcom.cc:486] [619431]Get Hcom Init Status: [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.275 [hcom_common.cc:85] [619431][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.588.286 [hcom_rpc.cc:1019] [619431]Get Hcom Init Status: [0] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:44.588.310 [hcom_base_funs.cc:36] [619431][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:44.588.327 [hcom_executor.cc:121] [619431][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.396 [api_impl.cc:1120] 619431 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.418 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.485 [stream.cc:2246] 619431 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.540 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.553 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.841 [npu_driver.cc:6125] 619431 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.901 [npu_driver.cc:6137] 619431 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1050, streamId=4, logicCq=1050, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.925 [npu_driver.cc:6189] 619431 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=4, logicCq=1050. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.942 [npu_driver.cc:5916] 619431 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec627000, addr=dfffec627000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.588.955 [stream.cc:653] 619431 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.589.015 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.589.030 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.589.350 [stream.cc:291] 619431 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.589.933 [stream.cc:688] 619431 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=3, streamResId=65535 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.589.951 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.589.963 [acl_resource_manager.cpp:816]619431 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.589.975 [api_impl.cc:1162] 619431 StreamCreate: Succ, flags=1536, stream_id=4, context=aaaaf46ef9e0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.590.002 [stream.cc:73] [619431]Construct stream by stream type success, ptr[0xaaaafebd78f0] ctx[0xaaaaf46ef9e0], stream id[4], cqId[4], logicCqid[1050] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.043 [api_impl.cc:1120] 619431 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.065 [npu_driver.cc:404] 619431 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.094 [stream.cc:2246] 619431 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.128 [stream_sqcq_manage.cc:147] 619431 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.140 [npu_driver.cc:882] 619431 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.307 [npu_driver.cc:6125] 619431 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.350 [npu_driver.cc:6137] 619431 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1051, streamId=5, logicCq=1051, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.368 [npu_driver.cc:6189] 619431 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=5, logicCq=1051. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.382 [npu_driver.cc:5916] 619431 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec606000, addr=dfffec606000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.394 [stream.cc:653] 619431 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619431,python3.9):2025-04-03-09:35:44.590.433 [devmm_svm.c:1972][ascend][curpid:619431,619431][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.450 [npu_driver.cc:6080] 619431 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.590.769 [stream.cc:291] 619431 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.425 [stream.cc:688] 619431 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=3, streamResId=65535 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.591.442 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.591.455 [acl_resource_manager.cpp:816]619431 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.467 [api_impl.cc:1162] 619431 StreamCreate: Succ, flags=1536, stream_id=5, context=aaaaf46ef9e0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.591.485 [stream.cc:73] [619431]Construct stream by stream type success, ptr[0xaaaafec53950] ctx[0xaaaaf46ef9e0], stream id[5], cqId[5], logicCqid[1051] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:44.591.641 [sal.cc:619] [619627]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.591.666 [hcom_executor.cc:587] [619627]MessageProcessThreadLoop start queType:[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.688 [api_impl.cc:2999] 619627 SetDevice: device_id=3, deviceMode=0. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:44.591.694 [sal.cc:619] [619628]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.701 [device.cc:306] 619627 Init: device mode has been already set, mode=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.591.722 [hcom_executor.cc:587] [619628]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.724 [runtime.cc:3393] 619627 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.740 [runtime.cc:3560] 619627 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.742 [api_impl.cc:2999] 619628 SetDevice: device_id=3, deviceMode=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:44.591.745 [hcom_executor.cc:34] [619431][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.740 [api_impl.cc:2999] 619629 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.752 [api_impl.cc:2987] 619627 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.756 [device.cc:306] 619628 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.772 [runtime.cc:5239] 619627 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.779 [runtime.cc:3393] 619628 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.782 [device.cc:306] 619629 Init: device mode has been already set, mode=0. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.784 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.790 [api_impl.cc:2992] 619627 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.794 [runtime.cc:3560] 619628 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.797 [runtime.cc:3393] 619629 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.793 [api_impl.cc:2999] 619630 SetDevice: device_id=3, deviceMode=0. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.808 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.808 [api_impl.cc:2987] 619628 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:44.591.812 [awatchdog_core.c:175](tid:619627) create watchdog for id : 65543, tid : 619627, timeout : 300s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.813 [runtime.cc:3560] 619629 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.818 [device.cc:306] 619630 Init: device mode has been already set, mode=0. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.834 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.833 [runtime.cc:5239] 619628 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.836 [api_impl.cc:3025] 619627 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.840 [api_impl.cc:2987] 619629 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.844 [runtime.cc:3393] 619630 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.858 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.857 [api_impl.cc:2992] 619628 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.858 [runtime.cc:5239] 619629 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.864 [runtime.cc:3560] 619630 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.878 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:44.591.877 [awatchdog_core.c:175](tid:619628) create watchdog for id : 65543, tid : 619628, timeout : 300s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.878 [api_impl.cc:2987] 619630 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.879 [api_impl.cc:2992] 619629 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.893 [api_impl.cc:3025] 619628 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.896 [runtime.cc:5239] 619630 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:44.591.905 [awatchdog_core.c:175](tid:619629) create watchdog for id : 65543, tid : 619629, timeout : 300s [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.913 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1126.73 msec. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.913 [api_impl.cc:2992] 619630 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.920 [api_impl.cc:3025] 619629 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:44.591.932 [awatchdog_core.c:175](tid:619630) create watchdog for id : 65543, tid : 619630, timeout : 300s [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.940 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.591.947 [api_impl.cc:3025] 619630 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [WARNING] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.591.972 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.015 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.012 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.060 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.092 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.111 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.07 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.233 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:44.592.261 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:44.592.339 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:44.592.369 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.592.409 [device.cpp:143]619631 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.436 [api_impl.cc:2999] 619631 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.448 [device.cc:306] 619631 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.459 [runtime.cc:3393] 619631 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.470 [runtime.cc:3560] 619631 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.480 [api_impl.cc:2987] 619631 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.491 [runtime.cc:5239] 619631 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.503 [api_impl.cc:2992] 619631 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:44.592.517 [awatchdog_core.c:175](tid:619631) create watchdog for id : 65543, tid : 619631, timeout : 300s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.592.529 [api_impl.cc:3025] 619631 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.592.540 [acl.cpp:115]619631 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.592.551 [acl.cpp:121]619631 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.592.570 [device.cpp:154]619631 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:44.592.603 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.185 msec. [WARNING] DEVICE(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:44.592.834 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 2 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.740.192 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.740.194 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.740.226 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:44.740.243 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.246 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.257 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.261 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.271 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.740.281 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.740.289 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.740.301 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:44.740.302 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.320 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.740.322 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.908.755 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:44.908.788 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:44.908.803 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:44.908.815 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.908.840 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:44.908.927 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:45.093.095 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:45.093.138 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.504 msec. [WARNING] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:45.093.154 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:45.093.180 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.093.327 [device.cpp:143]619757 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.356 [api_impl.cc:2999] 619757 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.369 [device.cc:306] 619757 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.382 [runtime.cc:3393] 619757 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.396 [runtime.cc:3560] 619757 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.408 [api_impl.cc:2987] 619757 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.420 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.436 [api_impl.cc:2992] 619757 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:45.093.452 [awatchdog_core.c:175](tid:619757) create watchdog for id : 65543, tid : 619757, timeout : 300s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.093.466 [api_impl.cc:3025] 619757 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.093.479 [acl.cpp:115]619757 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.093.491 [acl.cpp:121]619757 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.093.505 [device.cpp:154]619757 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.252.185 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.252.191 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.252.216 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.252.225 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.231 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.238 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.244 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.250 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.252.275 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.252.278 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.252.292 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.252.294 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.309 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.252.311 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.308.309 [device.cpp:231]619757 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.332 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.539 [api_c_soc.cc:70] 619757 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.308.604 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.621 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.635 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.646 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.659 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.676 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=0. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.308.700 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=0.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.308.733 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.748 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.761 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.773 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.783 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.308.796 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=1. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.308.968 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=1.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.309.000 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.015 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.027 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.053 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.066 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.078 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=2. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.309.367 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=2.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.309.399 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.414 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.430 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.440 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.450 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.461 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=4. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.309.903 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=4.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.309.934 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.948 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.962 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.975 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.309.990 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.004 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=5. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.310.440 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=5.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.310.472 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.486 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.498 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.509 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.521 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.310.532 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=6. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.311.284 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=6.) [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.311.315 [memory.cpp:559]619757 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.311.327 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.311.345 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.311.355 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.311.366 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.311.377 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=7. [INFO] DRV(619431,python3.9):2025-04-03-09:35:45.312.334 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=7.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.369 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.427 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.454 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.489 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.541 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.558 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.575 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.591 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.608 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.632 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.651 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.665 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.689 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.702 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.717 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.733 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.750 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.776 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.800 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.815 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.831 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.844 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.870 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.886 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.907 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.931 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.959 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.977 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.312.992 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.007 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.024 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.045 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.062 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.076 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.101 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.119 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.135 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.147 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.173 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.188 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.203 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.226 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.243 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.261 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.278 [runtime.cc:1664] 619757 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.292 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.313 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.331 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.347 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.381 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.407 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.435 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.451 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.465 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.484 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.506 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.517 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.529 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.543 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.558 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.584 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.597 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.609 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.624 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.638 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.665 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.676 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.691 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.707 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.720 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.744 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.757 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.770 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.786 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.801 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.829 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.848 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.861 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.876 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.890 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.911 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.923 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.313.936 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.239 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.274 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.316 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.330 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.344 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.359 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.380 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.418 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.430 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.442 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.458 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.478 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.511 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.524 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.537 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.555 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.576 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.614 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.626 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.639 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.658 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.676 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.715 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.728 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.741 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.754 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.775 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.814 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.827 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.842 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.855 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.870 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.905 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.921 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.935 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.949 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.314.968 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.006 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.019 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.033 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.049 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.064 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.102 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.116 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.128 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.141 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.168 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.207 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.219 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.230 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.243 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.259 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.295 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.306 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.318 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.334 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.358 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.392 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.406 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.420 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.432 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.447 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.482 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.493 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.505 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.520 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.542 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.579 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.593 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.604 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.616 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.631 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.667 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.691 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.703 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.719 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.739 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.776 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.786 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.799 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.813 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.830 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.869 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.882 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.893 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.905 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.920 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.954 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.968 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.982 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.315.995 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.010 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.044 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.057 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.072 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.089 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.105 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.143 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.155 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.200 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.222 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.244 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.285 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.300 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.314 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.330 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.346 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.381 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.394 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.405 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.459 [api_impl.cc:1990] 619757 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.805 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.316.839 [api_impl.cc:2664] 619757 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.352 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619419, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.378 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619423, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.396 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619427, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.414 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619435, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.430 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619439, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.446 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619443, num=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.461 [npu_driver.cc:3571] 619757 SetIpcMemPid: set ipc mem pid success, name=000973a700000000000003150300ffcc0003, pid[0]=619447, num=1 [2025-04-03 09:35:44.578804] [debug] [619431] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.578815] [debug] [619431] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.93628] [warn] [619757] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.93658] [debug] [619757] [lcal_sock_exchange.cpp:140] curRank: 3 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.93665] [debug] [619757] [lcal_sock_exchange.cpp:272] Client side 3 begin to connect [2025-04-03 09:35:45.93886] [debug] [619757] [lcal_sock_exchange.cpp:93] rank:3 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308275] [debug] [619757] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 3 rank :3 rankSize :8 [2025-04-03 09:35:45.308491] [debug] [619757] [lcal_comm.cpp:406] rank 3 devId: 3, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308505] [info] [619757] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308511] [info] [619757] [lcal_comm.cpp:242] rank 3/8 running devId:3 [2025-04-03 09:35:45.308557] [debug] [619757] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308594] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308724] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308991] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309390] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309927] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310464] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311307] [debug] [619757] [lcal_comm.cpp:340] 3 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312351] [debug] [619757] [lcal_comm.cpp:372] EnablePeerAccess succeed3 [2025-04-03 09:35:45.316420] [debug] [619757] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 3 [2025-04-03 09:35:45.316443] [debug] [619757] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.316825] [debug] [619757] [lcal_comm.cpp:455] peerMem[rank3], allocate finished. [2025-04-03 09:35:45.340235] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340249] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340258] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340265] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340272] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340279] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340284] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340291] [debug] [619757] [lcal_comm.cpp:472] rank : 3, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340298] [debug] [619757] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340474] [debug] [619757] [lcal_comm.cpp:557] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340790] [debug] [619757] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340800] [debug] [619757] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340807] [debug] [619757] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340813] [debug] [619757] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340830] [debug] [619757] [lcal_comm.cpp:515] rank 4 m[INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.340.883 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.342.031 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.342.050 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.343.114 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.343.135 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.343.917 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.343.933 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.357.999 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.358.021 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.359.126 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.359.142 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.008 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.022 [api_impl.cc:4286] 619757 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.598 [npu_driver.cc:3195] 619757 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.637 [api_impl.cc:1990] 619757 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.660 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.360.677 [memory.cpp:421]619757 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:45.360.862 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.360.905 [device.cpp:143]619757 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.925 [api_impl.cc:2999] 619757 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.937 [device.cc:306] 619757 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.948 [runtime.cc:3393] 619757 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.961 [runtime.cc:3560] 619757 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.974 [api_impl.cc:2987] 619757 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.360.988 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.361.004 [api_impl.cc:2992] 619757 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.361.018 [api_impl.cc:3025] 619757 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.361.029 [acl.cpp:115]619757 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.361.041 [acl.cpp:121]619757 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.361.056 [device.cpp:154]619757 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [WARNING] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:45.361.102 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.361.135 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.153 [adapter_rts.cc:198] [619757][hrtGetDeviceRefresh]deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.171 [externalinput.cc:2065] [619757]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.190 [externalinput.cc:490] [619757]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.207 [externalinput.cc:429] [619757]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.221 [externalinput.cc:543] [619757]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.235 [externalinput.cc:622] [619757]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.250 [externalinput.cc:714] [619757]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.266 [externalinput.cc:821] [619757]HCCL_IF_IP is not set [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.280 [externalinput.cc:877] [619757]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.293 [externalinput.cc:844] [619757]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.304 [externalinput.cc:728] [619757]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.316 [externalinput.cc:758] [619757]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.329 [externalinput.cc:806] [619757]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.340 [externalinput.cc:1508] [619757]HCCL_ALGO is not set [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.355 [externalinput.cc:1671] [619757][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.372 [externalinput.cc:462] [619757]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.386 [externalinput.cc:1519] [619757]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.399 [externalinput.cc:1554] [619757]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.361.512 [api_c_soc.cc:70] 619757 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.530 [externalinput.cc:1599] [619757]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.548 [externalinput.cc:1633] [619757]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.561 [externalinput.cc:1778] [619757]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.573 [externalinput.cc:1807] [619757]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.587 [externalinput.cc:515] [619757]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.599 [externalinput.cc:1841] [619757]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.612 [externalinput.cc:1929] [619757]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.636 [externalinput.cc:1949] [619757]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.649 [externalinput.cc:1969] [619757]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.671 [externalinput.cc:2005] [619757]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.685 [externalinput.cc:2036] [619757]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.698 [externalinput.cc:2209] [619757]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.713 [externalinput.cc:2178] [619757][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.724 [externalinput.cc:889] [619757]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.739 [externalinput.cc:914] [619757]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.752 [externalinput.cc:2240] [619757]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.765 [externalinput.cc:2308] [619757]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.782 [externalinput.cc:2339] [619757]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.798 [externalinput.cc:2296] [619757]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.813 [externalinput.cc:407] [619757]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:45.361.829 [externalinput.cc:672] [619757][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:45.361.872 [env_config.cc:127] [619757]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:45.361.887 [env_config.cc:131] [619757]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.901 [env_config.cc:302] [619757]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.913 [env_config.cc:302] [619757]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.924 [op_base.cc:1091] [619757]Entry-HcclCommInitRootInfoInner:ranks[8], rank[3], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.961 [op_base.cc:128] [619757][HcclGetDeviceId] deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.361.999 [op_base.cc:144] [619757][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.362.042 [rank_consistentcy_checker.cc:32] [619757][GetInstance] get deviceLogicId[3] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.362.079 [api_impl.cc:3100] 619757 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.362.092 [api_impl.cc:3111] 619757 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.362.108 [api_impl.cc:3130] 619757 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.362.310 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.362.342 [network_manager.cc:169] [619757]NetworkManager: devicePhyId[3], devicePhyId_[3] deviceLogicId_[3], hasBackup[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.374 [ra_host.c:1594]tid:619757,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.362.391 [adapter_hccp.cc:1450] [619757]set host socket whitelist status[0] success. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.426 [ra_host.c:205]tid:619757,ra_init(205) : Input parameters: phy_id[3], nic_position:[0] hdc_type:[6] [INFO] ROCE(619431,python3.9):2025-04-03-09:35:45.362.474 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.489 [ra_peer.c:776]tid:619757,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[3] start [INFO] ROCE(619431,python3.9):2025-04-03-09:35:45.362.544 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619431,python3.9):2025-04-03-09:35:45.362.599 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.628 [rs.c:327]tid:619757,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619431,python3.9):2025-04-03-09:35:45.362.775 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619431,python3.9):2025-04-03-09:35:45.362.797 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619431,python3.9):2025-04-03-09:35:45.362.811 [rs_ssl.c:1513]tid:619757,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.824 [rs_ssl.c:1529]tid:619757,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.362.863 [rs_epoll.c:692]tid:619757,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.017 [rs_epoll.c:487]tid:619771,rs_epoll_handle(487) : thread begin! thread_id:281469004017568, pid:619431, ppid:619411 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.062 [rs_epoll.c:415]tid:619771,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.067 [rs_epoll.c:713]tid:619757,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.078 [rs_epoll.c:511]tid:619771,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.079 [rs_epoll.c:623]tid:619774,rs_connect_handle(623) : thread begin! thread_id:281468995563424, pid:619431, ppid:619411 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.082 [rs.c:366]tid:619757,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.245000] ms [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.363.138 [rs_epoll.c:641]tid:619774,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.240 [rs.c:471]tid:619757,rs_init(471) : rs init success, chip_id[3] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.267 [ra_peer.c:813]tid:619757,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[3] succ [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.370.283 [adapter_hccp.cc:583] [619757]init ra success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.370.295 [network_manager.cc:222] [619757]NetworkManager nicDeploy[0] deviceLogicId_[3] devicePhyId_[3] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.315 [ra_host.c:1764]tid:619757,ra_get_ifnum(1764) : Input parameters: phy_id[3], nic_position:[0], is_all:[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.334 [ra_peer.c:859]tid:619757,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[3] start [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.355 [ra_peer.c:867]tid:619757,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[3] succ [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.370.384 [adapter_hccp.cc:1378] [619757][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.401 [ra_host.c:1814]tid:619757,ra_get_ifaddrs(1814) : Input parameters: phy_id[3], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.416 [ra_peer.c:874]tid:619757,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[3] start [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.431 [rs_socket.c:2743]tid:619757,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.446 [rs_socket.c:2743]tid:619757,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.461 [rs_socket.c:2743]tid:619757,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.473 [rs_socket.c:2749]tid:619757,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.485 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.498 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.509 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.519 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.531 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.543 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.556 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.571 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.583 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.595 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.608 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.621 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.632 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.646 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.657 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.668 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.679 [rs_socket.c:2749]tid:619757,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.692 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.703 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.715 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.726 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.736 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.759 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.772 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.785 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.796 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.807 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.819 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.831 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.843 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.856 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.873 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.885 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.896 [rs_socket.c:2749]tid:619757,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.908 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.920 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.932 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.944 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.954 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.963 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.975 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.987 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.370.999 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.012 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.023 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.035 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.048 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.060 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.072 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.082 [rs_socket.c:2751]tid:619757,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.095 [ra_peer.c:882]tid:619757,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[3] succ [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.117 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.140 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.155 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.168 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.183 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.197 [adapter_hccp.cc:1407] [619757][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.222 [sal.cc:365] [619757]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.235 [sal.cc:432] [619757]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.254 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:45.371.297 [ra_host.c:340]tid:619757,ra_socket_init_v1(340) : socket init:mode=0 phy_id=3 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.313 [adapter_hccp.cc:818] [619757]socket init v1 success, socketHandle[0xfffe940faba0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.325 [network_manager.cc:1218] [619757]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.338 [network_manager.cc:1257] [619757]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe940faba0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.394 [adapter_hal.cc:298] [619757]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.416 [sal.cc:593] [619757]IsHostUseDevNic[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.431 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.447 [network_manager.cc:169] [619757]NetworkManager: devicePhyId[4294967295], devicePhyId_[3] deviceLogicId_[3], hasBackup[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.489 [tsd_client.cpp:189][TsdCapabilityGet][tid:619757] TsdCapabilityGet Begin. [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.505 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.518 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.531 [process_mode_manager.cpp:1042][CapabilityGet][tid:619757] [ProcessModeManager] enter into CapabilityGet process deviceId[3] type[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.546 [adapter_tdt.cc:66] [619757]Get TsdCapability success. deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.559 [network_manager.cc:175] [619757][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:45.371.577 [network_manager.cc:99] [619757][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.592 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.604 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.614 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619757] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.635 [process_mode_manager.cpp:386][InitTsdClient][tid:619757] [TsdClient] tsd client has been inited before [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.653 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619757] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619431,python3.9):2025-04-03-09:35:45.371.671 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619757] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.684 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619757] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.695 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619757] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.710 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619757] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.732 [version_verify.cpp:114][SpecialFeatureCheck][tid:619757] VersionVerify: previous type[48], supported [INFO] TDT(619431,python3.9):2025-04-03-09:35:45.371.786 [hdc_client.cpp:154][TsdRecvData][tid:619757] recv timeout is:150000 ms [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.172 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.180 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.182 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.185 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.185 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.412.186 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.310 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.330 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.346 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.350 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.358 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.398 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.394 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.414 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.415 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.430 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.435 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.442 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.454 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.456 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.470 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.436 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.482 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.484 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.516 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.517 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.529 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.533 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.549 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.550 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.566 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.412.571 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.584 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.585 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.601 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.412.604 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.413.530 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.413.551 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.413.567 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.413.581 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.413.592 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.413.605 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.764.168 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.764.201 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.764.199 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.216 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:45.764.225 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.233 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.242 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.764.248 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.256 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.764.261 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.764.268 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.275 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:45.764.280 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:45.764.298 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.124.695 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:46.124.723 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:46.124.736 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:46.124.747 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.171 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.173 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.175 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.175 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.176 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.244.178 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.273 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.290 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.306 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.306 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.324 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.330 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.338 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.339 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.347 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.355 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.357 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.366 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.370 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.373 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.378 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.398 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.400 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.401 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.404 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.411 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.415 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.436 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.452 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.479 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.483 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.501 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.522 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.530 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.545 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.568 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.244.550 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.558 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.585 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.601 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.617 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.244.623 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.276.174 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.276.178 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.276.201 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.276.210 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.214 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.225 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.228 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.237 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.276.241 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.276.261 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.276.263 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.276.273 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.279 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.276.287 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.788.168 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.788.168 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.788.194 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:46.788.204 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.208 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.217 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.220 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.229 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.788.233 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.788.240 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.788.245 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:46.788.252 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.259 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:46.788.265 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.172 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.174 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.176 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.177 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.176 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.076.175 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.273 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.289 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.297 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.302 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.316 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.322 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.326 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.332 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.339 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.359 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.363 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.369 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.381 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.390 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.418 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.433 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.396 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.398 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.402 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.438 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.444 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.451 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.461 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.468 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.478 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.484 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.509 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.535 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.553 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.602 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.609 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.076.614 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.647 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.650 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.659 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.076.662 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.300.170 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.300.174 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.300.199 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.300.206 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.212 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.220 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.225 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.232 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.300.238 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.300.247 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.300.250 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.300.259 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.264 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.300.286 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.731.477 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.731.478 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619757] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.731.514 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619757] [TsdClient] DeviceMsgProc recvMsg realDeviceId[3] msgType[49] localDevId[3] rspCode[0]heterogeneousSubPid[31184], tsdSupportLevel_[63] [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.731.531 [hdc_client.cpp:172][TsdRecvData][tid:619757] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.731.545 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619757] OpenSubProc success stype:0, pid:31184 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.731.558 [adapter_tdt.cc:39] [619757]Open TsdProcess success. deviceLogicId[3] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.731.568 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.731.571 [network_manager.cc:118] [619757][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31184], devicePhyId_[3], deviceLogicId_[3], hasBackup[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.731.593 [network_manager.cc:180] [619757][NetworkManager][Init]open tsd by process success, devicePhyId[3], deviceLogicId_[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.731.636 [ra_host.c:205]tid:619757,ra_init(205) : Input parameters: phy_id[3], nic_position:[1] hdc_type:[18] [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.731.651 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619431,python3.9):2025-04-03-09:35:47.731.669 [devdrv_hccl.c:103][ascend][curpid:619431,619757][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619431) [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.731.683 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.731.698 [ra_hdc.c:580]tid:619757,ra_hdc_init(580) : hdc init start! logic id is 3, phy id is 3, hdc_type is 18 [INFO] DRV(619431,python3.9):2025-04-03-09:35:47.731.856 [hdc_core.c:1982][ascend][curpid:619431,619757][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4266; pid=619431) [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.732.102 [ra_hdc.c:616]tid:619757,ra_hdc_init(616) : hdc init OK! phy_id[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.735.519 [adapter_hccp.cc:583] [619757]init ra success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.735.533 [network_manager.cc:222] [619757]NetworkManager nicDeploy[1] deviceLogicId_[3] devicePhyId_[3] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.735.555 [topoinfo_detect.cc:534] [619757]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.735.571 [ra_host.c:1764]tid:619757,ra_get_ifnum(1764) : Input parameters: phy_id[3], nic_position:[1], is_all:[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.735.945 [adapter_hccp.cc:1538] [619757][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.735.958 [ra_host.c:1814]tid:619757,ra_get_ifaddrs(1814) : Input parameters: phy_id[3], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.300 [adapter_hccp.cc:1568] [619757][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth3] ip[29.105.51.138%eth3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.328 [adapter_hccp.cc:1568] [619757][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth3] ip[fe80::2287:ecff:fe02:45b4%eth3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.343 [topoinfo_detect.cc:577] [619757]select AF_INET family as device socket family. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.357 [topoinfo_detect.cc:756] [619757]no backup device ip: use 0 as device ip. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.367 [topoinfo_detect.cc:294] [619757][HCCL_TRACE]SetupAgent rankNum[8], rank[3], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[3], phydevId[3], deviceIp[29.105.51.138] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.398 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.413 [hccl_socket.cc:516] [619757][Get][NicHandleInfo]phyId[3], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.428 [hccl_socket.cc:217] [619757][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281468870831008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.736.442 [adapter_hccp.cc:1050] [619757]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.458 [ra_host.c:759]tid:619757,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[3], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.496 [rs_socket.c:1810]tid:619757,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.511 [rs_socket.c:1819]tid:619757,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619431,python3.9):2025-04-03-09:35:47.736.524 [rs_socket.c:132]tid:619757,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.540 [rs_socket.c:1880]tid:619757,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.796 [rs_socket.c:1577]tid:619774,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:44939, fd:54 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.838 [rs_drv_socket.c:59]tid:619774,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:54 begin [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.925 [rs_drv_socket.c:98]tid:619774,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:44939, server addr 192.168.0.44:60008, fd:54 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.948 [rs_socket.c:1427]tid:619774,rs_conn_cost_time(1427) : socket [54] connect success! cost [0.408000] ms [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.736.989 [rs_socket.c:1392]tid:619774,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.737.633 [topoinfo_exchange_agent.cc:212] [619757]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.737.701 [topoinfo_exchange_agent.cc:236] [619757]local rank[3] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.737.726 [topoinfo_exchange_agent.cc:43] [619757]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.737.744 [topoinfo_exchange_agent.cc:345] [619757]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.737.838 [topoinfo_exchange_agent.cc:77] [619757]topo exchange client send rank basic info success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.801.028 [api_impl.cc:2096] 619757 HostMalloc: size=2485 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.801.074 [api_error.cc:1260] 619757 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.801.367 [api_error.cc:1278] 619757 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.386 [topoinfo_exchange_agent.cc:80] [619757]topo exchange client get rank basic info success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.442 [topoinfo_detect.cc:303] [619757]topo detect completed. myrank[3], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.475 [config.cc:198] [619757]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.499 [config.cc:210] [619757]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.515 [config.cc:210] [619757]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.529 [config.cc:210] [619757]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.542 [config.cc:210] [619757]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.554 [config.cc:210] [619757]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.567 [config.cc:210] [619757]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.579 [config.cc:210] [619757]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.591 [config.cc:210] [619757]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.801.606 [adapter_hccp.cc:1067] [619757]ra socket batch close fdhandle[0xfffe94054bf0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.801.620 [ra_host.c:796]tid:619757,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[3], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.801.639 [rs_socket.c:1937]tid:619757,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:54, state:9 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.812.181 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.812.185 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.812.200 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.812.206 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.220 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.223 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.234 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.237 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.812.246 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.812.249 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.812.259 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.812.262 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.272 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.812.275 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.848.198 [adapter_hccp.cc:1099] [619757]ra socket batch close success,take time [46580]us [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:47.848.219 [hccl_socket.cc:232] [619757][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.848.238 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.256 [ra_host.c:404]tid:619757,ra_socket_deinit(404) : Input parameters: phy_id[3] family[2] local_ip[192.168.0.44] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.273 [rs.c:1629]tid:619757,rs_socket_deinit(1629) : rs socket deinit start, phy_id:3 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.287 [rs.c:1641]tid:619757,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.307 [rs.c:1653]tid:619757,rs_socket_deinit(1653) : socket deinit success, phy_id:3, local_ip:192.168.0.44 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.848.322 [network_manager.cc:1233] [619757]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.848.337 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.352 [ra_host.c:248]tid:619757,ra_deinit(248) : Input parameters: phy_id[3], nic_position:[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.365 [ra_peer.c:821]tid:619757,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[3] start [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.385 [rs.c:1783]tid:619757,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619431,python3.9):2025-04-03-09:35:47.848.390 [rs_epoll.c:523]tid:619771,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:52 [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.848.412 [rs_epoll.c:544]tid:619771,rs_epoll_handle(544) : QUIT thread_id:281469004017568, pid:619431, read num:8 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.172 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.173 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.173 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.175 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.181 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:47.908.180 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.300 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.301 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.329 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.332 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.336 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.341 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.349 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.354 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.360 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.362 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.367 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.371 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.377 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.383 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.372 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.393 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.400 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.401 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.407 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.433 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.439 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.440 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.457 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.527 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.529 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.517 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.534 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.539 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.543 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.563 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:47.908.557 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.566 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.577 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.580 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.588 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:47.908.591 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.947.449 [rs_epoll.c:679]tid:619774,rs_connect_handle(679) : QUIT thread_id:281468995563424, pid:619431 [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.948.685 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.948.707 [rs.c:1810]tid:619757,rs_deinit(1810) : rs_deinit chip_id[3] ok [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.948.722 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.948.738 [ra_peer.c:852]tid:619757,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[3] succ [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.948.751 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.948.781 [network_manager.cc:409] [619757]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.948.797 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.948.812 [ra_host.c:248]tid:619757,ra_deinit(248) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.948.823 [ra_hdc.c:651]tid:619757,ra_hdc_deinit(651) : hdc deinit start! phy_id[3] [INFO] DRV(619431,python3.9):2025-04-03-09:35:47.948.963 [hdc_client.c:417][ascend][curpid:619431,619757][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4266) [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.949.039 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:47.949.053 [ra_hdc.c:685]tid:619757,ra_hdc_deinit(685) : hdc deinit OK! phy_id[3] [INFO] ROCE(619431,python3.9):2025-04-03-09:35:47.949.067 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.091 [tsd_client.cpp:189][TsdCapabilityGet][tid:619757] TsdCapabilityGet Begin. [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.104 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.118 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.129 [process_mode_manager.cpp:1042][CapabilityGet][tid:619757] [ProcessModeManager] enter into CapabilityGet process deviceId[3] type[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:47.949.142 [adapter_tdt.cc:66] [619757]Get TsdCapability success. deviceLogicId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.154 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.166 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.178 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619757] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.193 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619757] add close subproc:31184, proctype:0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.206 [version_verify.cpp:114][SpecialFeatureCheck][tid:619757] VersionVerify: previous type[60], supported [INFO] TDT(619431,python3.9):2025-04-03-09:35:47.949.235 [hdc_client.cpp:154][TsdRecvData][tid:619757] recv timeout is:150000 ms [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.201.863 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619757] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.201.887 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619757] [TsdClient] DeviceMsgProc recvMsg realDeviceId[3] msgType[61] localDevId[3] rspCode[0]heterogeneousSubPid[31184], tsdSupportLevel_[63] [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.201.904 [hdc_client.cpp:172][TsdRecvData][tid:619757] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.201.917 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619757] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.201.933 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619757] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.201.945 [adapter_tdt.cc:52] [619757]Open TsdProcess success. deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.201.955 [network_manager.cc:405] [619757][DeInit]finish CloseHccpProcess, phy_id[3], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.201.968 [network_manager.cc:409] [619757]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.201.980 [topoinfo_detect.cc:555] [619757]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.201.991 [topoinfo_detect.cc:321] [619757]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.002 [topoinfo_detect.cc:219] [619757]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.025 [rank_consistentcy_checker.cc:32] [619757][GetInstance] get deviceLogicId[3] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:48.202.037 [rank_consistentcy_checker.cc:144] [619757][Record][CannVersion] version information str is empty. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.048 [op_base.cc:2231] [619757]rank table is null, rankTableCrc is 0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.071 [op_base.cc:2219] [619757]params.id.internal [hccl-619431-0-1743644148-hccl_world_group] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.080 [op_base.cc:993] [619757]rootInfo[hccl-619431-0-1743644148-hccl_world_group], params.logiceDevice[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.092 [hccl_comm.cc:98] [619757]hcclComm init workmode [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.101 [hccl_comm.cc:954] [619757]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.953 [hccl_comm.cc:1365] [619757]RegistTaskAbortHandler begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.202.974 [task_abort_handler.cc:126] [619757]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.202.988 [api_impl.cc:4192] 619757 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.002 [runtime.cc:4698] 619757 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.051 [adapter_rts.cc:42] [619757]open so: libruntime.so [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.287 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.319 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.334 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.355 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.368 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.386 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.400 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.417 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.429 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.448 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.463 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.480 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.494 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.511 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.524 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.558 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.573 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.590 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.601 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.618 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.631 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.649 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.660 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.677 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.692 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.708 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.721 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.739 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.751 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.770 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.784 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.800 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.811 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.827 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.841 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.859 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.873 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.891 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.904 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.936 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.950 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.967 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.203.980 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.203.999 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.011 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.024 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.037 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.055 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.069 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.087 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.103 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.121 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.133 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.148 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.180 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.204 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.248 [hccl_communicator_attrs.cc:407] [619757]module[0] contains [8]devices [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.260 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.273 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.286 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.297 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.310 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.320 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.333 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.357 [hccl_communicator_attrs.cc:411] [619757]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.369 [hccl_communicator_attrs.cc:425] [619757]different module contains different numbers of cards:[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.382 [hccl_communicator_attrs.cc:426] [619757]different module contains different type of cards:[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.426 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.449 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.465 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.488 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.502 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.520 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.535 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.554 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.567 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.587 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.601 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.619 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.633 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.651 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.664 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.683 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.697 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.716 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.730 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.749 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.762 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.789 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.801 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.820 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.835 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.857 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.871 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.891 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.904 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.920 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.934 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.952 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.204.968 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.204.987 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.003 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.022 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.038 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.055 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.069 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.087 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.100 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.117 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.132 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.150 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.165 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.183 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.209 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.227 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.239 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.257 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.271 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.289 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.301 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.319 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.331 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.348 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.361 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.380 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.393 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.410 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.424 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.446 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.458 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.476 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.492 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.513 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.527 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.546 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.561 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.579 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.592 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.618 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.631 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.648 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.662 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.677 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.690 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.708 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.721 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.739 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.755 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.772 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.786 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.804 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.816 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.834 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.847 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.867 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.880 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.896 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.908 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.930 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.944 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.962 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.205.976 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.205.994 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.013 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.032 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.046 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.063 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.077 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.094 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.107 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.126 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.141 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.160 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.174 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.192 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.206 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.225 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.239 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.257 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.274 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.288 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.301 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.317 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.332 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.350 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.365 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.383 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.405 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.423 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.439 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.458 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.471 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.488 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.500 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.518 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.531 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.551 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.564 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.582 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.595 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.611 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.625 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.641 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.655 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.673 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.688 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.707 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.722 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.740 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.752 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.767 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.781 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.804 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.819 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.837 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.850 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.867 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.880 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.899 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.913 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.931 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.944 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.959 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.206.974 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.206.990 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.005 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.021 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.035 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.054 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.069 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.087 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.104 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.121 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.135 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.151 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.183 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.214 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.230 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.246 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.263 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.281 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.299 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.315 [hccl_communicator_attrs.cc:156] [619757][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.333 [hccl_communicator_attrs.cc:1021] [619757][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.348 [hccl_communicator.cc:1440] [619757][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.363 [hccl_communicator.cc:1450] [619757][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.377 [externalinput.cc:1445] [619757]InitParsesProtocol start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.392 [externalinput.cc:1462] [619757]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:48.207.405 [externalinput.cc:1411] [619757]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.427 [externalinput.cc:1413] [619757]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.440 [externalinput.cc:1467] [619757]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.482 [config.cc:557] [619757][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.498 [hccl_communicator.cc:668] [619757][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.520 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.539 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.553 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=0. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.580 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=0.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.592 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.605 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.620 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.633 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=1. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.647 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=1.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.659 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.673 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.690 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.703 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=2. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.718 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=2.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.731 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.747 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.761 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.775 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=4. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.789 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=4.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.800 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.813 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.828 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.841 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=5. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.856 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=5.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.869 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.881 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.896 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.909 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=6. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.923 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=6.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.207.936 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.947 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.962 [api_impl.cc:3520] 619757 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.207.981 [api_impl.cc:3168] 619757 EnableP2P: Enable P2P devIdDes=3, phyIdSrc=7. [INFO] DRV(619431,python3.9):2025-04-03-09:35:48.207.999 [devdrv_pcie.c:888][ascend][curpid:619431,619757][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=3; peer_phy_devid=7.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.015 [p2p_mgmt.cc:74] [619757]enable p2p: local logic id:3, remote physic id:7. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.039 [adapter_hal.cc:298] [619757]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.061 [sal.cc:593] [619757]IsHostUseDevNic[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.107 [adapter_hal.cc:298] [619757]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.126 [sal.cc:593] [619757]IsHostUseDevNic[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.139 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.150 [network_manager.cc:169] [619757]NetworkManager: devicePhyId[4294967295], devicePhyId_[3] deviceLogicId_[3], hasBackup[0] [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.181 [tsd_client.cpp:189][TsdCapabilityGet][tid:619757] TsdCapabilityGet Begin. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.195 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.209 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.222 [process_mode_manager.cpp:1042][CapabilityGet][tid:619757] [ProcessModeManager] enter into CapabilityGet process deviceId[3] type[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.238 [adapter_tdt.cc:66] [619757]Get TsdCapability success. deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.248 [network_manager.cc:175] [619757][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:48.208.265 [network_manager.cc:99] [619757][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.280 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.294 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619757] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.305 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619757] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.319 [process_mode_manager.cpp:386][InitTsdClient][tid:619757] [TsdClient] tsd client has been inited before [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.332 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619757] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619431,python3.9):2025-04-03-09:35:48.208.346 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619757] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.358 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619757] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.370 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619757] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.385 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619757] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.208.429 [hdc_client.cpp:154][TsdRecvData][tid:619757] recv timeout is:150000 ms [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.324.177 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.324.189 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.324.205 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.324.214 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.220 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.227 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.232 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.239 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.324.246 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.324.251 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.324.259 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.324.263 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.273 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.324.276 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.168 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.182 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.184 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.192 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.193 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.740.193 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.280 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.297 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.314 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.319 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.325 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.336 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.338 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.351 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.354 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.355 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.363 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.373 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.376 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.388 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.388 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.390 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.408 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.415 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.415 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.429 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.437 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.445 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.451 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.460 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.469 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.477 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.479 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.501 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.507 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.531 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.536 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.596 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.604 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.740.609 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.636 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.740.674 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.836.179 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.836.179 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.836.203 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:48.836.211 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.215 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.224 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.227 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.236 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.836.239 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.836.248 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.836.250 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:48.836.263 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.266 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.836.277 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:48.940.713 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.940.744 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.940.758 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:48.940.774 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.348.174 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.348.179 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.348.193 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.348.199 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.205 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.211 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.216 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.223 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.348.226 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.348.233 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.348.236 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.348.244 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.248 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.348.258 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.178 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.179 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.177 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.184 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.183 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.572.184 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.290 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.310 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.324 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.325 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.336 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.344 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.349 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.356 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.360 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.365 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.368 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.381 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.383 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.397 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.400 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.401 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.410 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.427 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.435 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.437 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.443 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.450 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.460 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.468 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.484 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.500 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.504 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.515 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.520 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.523 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.533 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.539 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.571 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.572.614 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.625 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.572.667 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.860.166 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.860.180 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.860.187 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:49.860.196 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.199 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.208 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.212 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.220 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.860.224 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.860.235 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.860.237 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:49.860.246 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.250 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:49.860.258 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] TDT(619431,python3.9):2025-04-03-09:35:50.310.657 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619757] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.310.663 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:50.310.678 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619757] [TsdClient] DeviceMsgProc recvMsg realDeviceId[3] msgType[49] localDevId[3] rspCode[0]heterogeneousSubPid[31203], tsdSupportLevel_[63] [INFO] TDT(619431,python3.9):2025-04-03-09:35:50.310.693 [hdc_client.cpp:172][TsdRecvData][tid:619757] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:50.310.705 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619757] OpenSubProc success stype:0, pid:31203 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.310.721 [adapter_tdt.cc:39] [619757]Open TsdProcess success. deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.310.732 [network_manager.cc:118] [619757][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31203], devicePhyId_[3], deviceLogicId_[3], hasBackup[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.310.741 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.310.745 [network_manager.cc:180] [619757][NetworkManager][Init]open tsd by process success, devicePhyId[3], deviceLogicId_[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:50.310.795 [ra_host.c:205]tid:619757,ra_init(205) : Input parameters: phy_id[3], nic_position:[1] hdc_type:[18] [INFO] ROCE(619431,python3.9):2025-04-03-09:35:50.310.845 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619431,python3.9):2025-04-03-09:35:50.310.862 [devdrv_hccl.c:103][ascend][curpid:619431,619757][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619431) [INFO] ROCE(619431,python3.9):2025-04-03-09:35:50.310.894 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:50.310.906 [ra_hdc.c:580]tid:619757,ra_hdc_init(580) : hdc init start! logic id is 3, phy id is 3, hdc_type is 18 [INFO] DRV(619431,python3.9):2025-04-03-09:35:50.311.190 [hdc_core.c:1982][ascend][curpid:619431,619757][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4273; pid=619431) [INFO] HCCP(619431,python3.9):2025-04-03-09:35:50.311.484 [ra_hdc.c:616]tid:619757,ra_hdc_init(616) : hdc init OK! phy_id[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.159 [adapter_hccp.cc:583] [619757]init ra success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.175 [network_manager.cc:222] [619757]NetworkManager nicDeploy[1] deviceLogicId_[3] devicePhyId_[3] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.190 [hccl_communicator_attrs.cc:1307] [619757][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.203 [hccl_communicator_attrs.cc:1275] [619757]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.219 [hccl_communicator.cc:2259] [619757][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.238 [adapter_hccp.cc:1650] [619757]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.452 [adapter_hccp.cc:2343] [619757]Get available Vnic info success, deviceIdType[0] deviceId[0x3] Vnic ip[192.168.2.196] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.626 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:50.315.691 [ra_host.c:280]tid:619757,ra_socket_init(280) : socket init:mode=1 phy_id=3 family=2 ip=192.168.2.196 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.715 [adapter_hccp.cc:806] [619757]socket init success, ip[3288508608] device id[3], socketHandle[0xfffe940ffbb0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.733 [network_manager.cc:692] [619757]devicePhyId[3], ip[192.168.2.196] socket init OK [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.315.746 [network_manager.cc:432] [619757][Start][Vnic]trying to listen on ip[192.168.2.196] port[16666]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:50.315.768 [ra_host.c:875]tid:619757,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[3], local_ip[192.168.2.196], port[16666] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.104 [network_manager.cc:448] [619757][Start][Vnic]Listen on ip[192.168.2.196], port[16666] sucess, devPhyId[3], devLogicId[3], isAutoPort[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.125 [network_manager.cc:451] [619757]NetworkManager devicePhyId_[3] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.140 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.154 [hccl_socket.cc:106] [619757][HcclSocket][Listen] device[3] listens on ip[192.168.2.196] port[16666] success, socketType[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.183 [hccl_socket_manager.cc:56] [619757][Init][Server]ip[192.168.2.196] port[16666] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.206 [hccl_communicator.cc:1548] [619757][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.196] port[16666] success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.224 [hccl_communicator_attrs.cc:1307] [619757][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.238 [hccl_communicator_attrs.cc:1275] [619757]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.255 [hccl_communicator.cc:1565] [619757]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.269 [network_manager.cc:69] [619757]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.286 [hccl_communicator.cc:2028] [619757]profiling config information:options[], mode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.557 [adapter_trace.cc:21] [619757]Call TraceCreate, return value[0] [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.316.583 [atrace_api.c:61](tid:619757) AtraceCreateWithAttr start. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.316.667 [trace_rb_log.c:45](tid:619757) init ring buffer time finished, realTime 1743644150316659857, monotonicTime 4211952659247, freq 99997 kHz, minutesWest 480 minutes [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.316.687 [trace_rb_log.c:137](tid:619757) [HCCL_619757_3] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.316.708 [tracer_mgr_operate.c:66](tid:619757) create object HCCL_619757_3 success, exitSave(true), noLock(false). [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.316.721 [atrace_api.c:63](tid:619757) AtraceCreateWithAttr end. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.732 [adapter_trace.cc:70] [619757]Call TraceCreateWithAttr, return value[0xaaaae2c52708], Params: objName[HCCL_619757_3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.788 [rank_consistentcy_checker.cc:32] [619757][GetInstance] get deviceLogicId[3] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.316.837 [api_impl.cc:1486] 619757 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.316.874 [npu_driver.cc:4604] 619757 GetMaxStreamAndTask: deviceId=3, tsId=0, streamCount=1984. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.893 [task_exception_handler.cc:1212] [619757]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.316.916 [hccl_communicator_attrs.cc:1099] [619757][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.316.973 [api_impl.cc:4219] 619757 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.317.003 [command_handle.cc:25] [619757][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.317.019 [command_handle.cc:99] [619757][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.317.123 [profiler_manager_impl.cc:76] [619757][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.317.143 [hccl_communicator.cc:2193] [619757][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.317.157 [hccl_communicator.cc:803] [619757]Will use ffts mode. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.317.186 [api_impl.cc:3138] 619757 GetDeviceIndexByPhyId: get Index by PhyId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.317.205 [api_impl.cc:3149] 619757 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.317.218 [api_impl.cc:3157] 619757 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.317.233 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.318.305 [api_impl.cc:3476] 619757 GetDeviceInfo: get device info, deviceId=3 [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:50.318.367 [tbe_vector_reduce.cc:508] [619757]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.103 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.279 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.319.300 [dispatcher.cc:157] [619757][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.319 [api_impl.cc:3138] 619757 GetDeviceIndexByPhyId: get Index by PhyId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.334 [api_impl.cc:3149] 619757 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.345 [api_impl.cc:3157] 619757 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.357 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.319.383 [api_impl.cc:1990] 619757 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.320.233 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.320.313 [api_impl.cc:1990] 619757 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.320.338 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.320.372 [api_impl.cc:1990] 619757 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.320.392 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.436 [opexecounter.cc:71] [619757]alloc counter mem resource. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.494 [hccl_communicator_attrs.cc:1368] [619757][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.533 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.556 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.577 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.595 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.612 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.629 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.647 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.665 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.681 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.696 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.710 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.727 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.744 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.760 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.774 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.788 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.803 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.816 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.831 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.844 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.857 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.884 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.899 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.913 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.927 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.941 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.953 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.967 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.981 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.320.994 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.008 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.025 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.039 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.056 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.070 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.083 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.097 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.110 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.124 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.137 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.152 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.165 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.178 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.203 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.219 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.232 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.249 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.263 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.279 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.293 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.306 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.319 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.333 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.346 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.358 [alg_configurator.cc:68] [619757][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.372 [alg_configurator.cc:241] [619757][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.385 [alg_configurator.cc:134] [619757]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.405 [alg_configurator.cc:449] [619757][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.420 [alg_configurator.cc:42] [619757]alg configurator init success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.460 [topo_info_extractor.cc:66] [619757]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[3], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.484 [topo_info_extractor.cc:155] [619757][SetRankInfo]rankData_: userRank[3], devicePhyId[3], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.511 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.552 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.584 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.605 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.625 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.656 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.688 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.712 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.728 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.745 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.761 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.779 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.321.795 [api_error.cc:3478] 619757 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.816 [adapter_rts.cc:940] [619757][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.835 [topo_info_extractor.cc:348] [619757]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.321.985 [topo_info_extractor.cc:888] [619757]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[3], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.018 [topo_info_extractor.cc:585] [619757][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.039 [topo_info_extractor.cc:646] [619757][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.057 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.074 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.090 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.105 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/1; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.122 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.138 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.154 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.171 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.193 [topo_info_extractor.cc:689] [619757]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[3], userRankSize[8], plane size[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.207 [topo_info_extractor.cc:697] [619757][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.220 [topo_info_extractor.cc:734] [619757]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 3/3/0/; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.232 [topo_info_extractor.cc:739] [619757]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[3], userRankSize[8], plane size[1] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.245 [topo_info_extractor.cc:585] [619757][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.262 [topo_info_extractor.cc:646] [619757][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.279 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.303 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.317 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.335 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/1; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.351 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.366 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.382 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.397 [topo_info_extractor.cc:685] [619757]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.409 [topo_info_extractor.cc:689] [619757]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[3], userRankSize[8], plane size[8] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.452 [topo_info_extractor.cc:790] [619757]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId3/3; ] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.470 [topo_info_extractor.cc:519] [619757]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.322.600 [api_impl.cc:1990] 619757 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.322.632 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.679 [hccl_impl.cc:59] [619757]start hccl resources build:get deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.322.814 [hccl_impl.cc:177] [619757]hcclImpl init success. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:50.322.856 [hccl_aiv.cc:279] [619757][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.330 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.407 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.527 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.547 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.570 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.594 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.618 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.719 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.735 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.753 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.771 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.794 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.898 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.914 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.931 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.947 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.325.968 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.062 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.079 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.093 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.111 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.132 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.229 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.248 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.266 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.282 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.332 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.427 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.444 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.458 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.473 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.494 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.586 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.602 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.618 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.634 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.655 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.749 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.765 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.781 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.796 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.815 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.905 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.921 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.940 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.956 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.326.977 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.069 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.086 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.101 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.118 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.138 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.240 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.258 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.273 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.287 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.309 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.404 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.421 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.436 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.450 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.468 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.563 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.579 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.594 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.610 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.628 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.720 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.738 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.752 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.767 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.785 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.877 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.894 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.911 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.926 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.327.946 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.040 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.058 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.082 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.097 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.118 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.233 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.255 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.272 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.288 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.307 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.399 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.416 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.431 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.445 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.463 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.555 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.573 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.588 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.603 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.622 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.714 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.731 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.746 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.761 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.780 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.878 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.894 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.908 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.931 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.328.952 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.046 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.063 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.079 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.093 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.112 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.206 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.223 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.240 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.255 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.275 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.367 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.383 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.398 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.412 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.433 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.524 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.542 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.557 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.572 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.590 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.680 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.696 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.710 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.724 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.751 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.847 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.867 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.882 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.896 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.329.915 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.007 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.024 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.039 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.053 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.071 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.163 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.180 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.194 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.209 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.227 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.322 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.339 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.353 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.367 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.387 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.481 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.497 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.512 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.526 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.546 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.637 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.664 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.679 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.694 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.713 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.813 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.831 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.845 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.859 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.877 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.969 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.330.985 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.000 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.014 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.032 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.122 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.141 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.156 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.170 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.187 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.279 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.295 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.311 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.326 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.343 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.434 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.452 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.474 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.488 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.507 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.600 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.616 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.630 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.645 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.662 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.753 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.771 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.786 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.801 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.819 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.913 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.931 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.947 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.961 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.331.982 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.072 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.090 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.104 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.119 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.136 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.242 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.266 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.282 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.305 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.327 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.416 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.434 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.449 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.463 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.484 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.575 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.592 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.607 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.621 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.642 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.739 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.755 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.769 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.784 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.804 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.895 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.913 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.927 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.943 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.332.962 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.055 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.071 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.085 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.099 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.120 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.218 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.234 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.248 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.262 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.284 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.386 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.403 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.417 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.432 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.452 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.542 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.559 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.573 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.588 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.607 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.698 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.715 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.729 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.745 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.768 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.859 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.877 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.891 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.904 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.333.923 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.020 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.046 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.063 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.078 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.099 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.189 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.206 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.221 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.235 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.255 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.346 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.364 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.378 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.394 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.414 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.506 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.525 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.540 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.555 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.574 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.667 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.686 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.703 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.718 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.736 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.827 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.844 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.866 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.881 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.900 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.334.988 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.006 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.021 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.036 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.055 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.148 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.166 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.181 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.195 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.214 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.306 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.325 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.339 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.353 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.371 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.459 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.478 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.492 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.507 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.525 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.614 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.631 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.646 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.661 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.687 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.786 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.803 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.818 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.832 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.850 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.937 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.956 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.972 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.335.987 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.004 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.093 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.110 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.125 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.140 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.171 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.267 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.287 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.301 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.316 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.334 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.428 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.446 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.461 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.475 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.494 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.589 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.614 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.629 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.644 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.662 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.753 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.774 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.788 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.805 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.823 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.914 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.934 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.949 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.963 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.336.981 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.075 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.092 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.108 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.123 [runtime.cc:1673] 619757 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.142 [elf.cc:566] 619757 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.232 [runtime.cc:1688] 619757 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.250 [runtime.cc:1953] 619757 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.266 [runtime.cc:1897] 619757 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619431,python3.9):2025-04-03-09:35:50.337.296 [devmm_svm.c:1972][ascend][curpid:619431,619757][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.337.313 [hdc.cc:83] [619757][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.327 [api_impl.cc:1990] 619757 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.356 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.963 [api_impl.cc:1990] 619757 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.337.985 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.006 [api_impl.cc:2664] 619757 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619431,python3.9):2025-04-03-09:35:50.338.170 [devmm_svm.c:1972][ascend][curpid:619431,619757][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.189 [hdc.cc:83] [619757][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.202 [api_impl.cc:1990] 619757 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.219 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.245 [api_impl.cc:2096] 619757 HostMalloc: size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.267 [api_error.cc:1260] 619757 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.578 [api_impl.cc:1990] 619757 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.654 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.676 [api_impl.cc:2664] 619757 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.817 [api_impl.cc:1990] 619757 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.837 [api_error.cc:1218] 619757 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.338.854 [api_impl.cc:2664] 619757 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.882 [peterson_lock.cc:69] [619757][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.896 [peterson_lock.cc:47] [619757][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.929 [hccl_comm.cc:129] [619757]hcclCommInitInfo:commId[hccl-619431-0-1743644148-hccl_world_group], rank[3], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[3], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.950 [hccl_comm.cc:175] [619757][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.967 [topo_matcher.cc:528] [619757][SetAivMode]AivMode is set to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.338.983 [topo_matcher.cc:540] [619757][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.001 [hcom_plugin.cc:349] [619757][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.339.024 [runtime.cc:5239] 619757 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.038 [adapter_rts.cc:198] [619757][hrtGetDeviceRefresh]deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.051 [hcom_common.cc:85] [619757][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.077 [op_base.cc:128] [619757][HcclGetDeviceId] deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.093 [hccl_comm.cc:1296] [619757]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffebcf9ccb8] topoSize[2] [INFO] GE(619431,python3.9):2025-04-03-09:35:50.339.117 [hcom_topo_info.cc:24]619757 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.131 [hcom_plugin.cc:371] [619757][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.157 [topoinfo_detect.cc:555] [619757]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.171 [topoinfo_detect.cc:321] [619757]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.183 [topoinfo_detect.cc:219] [619757]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.213 [op_base.cc:1050] [619757][InitCommRootInfo]Init success, rankNum[8], rank[3], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.231 [profiler_base.cc:342] [619757]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.248 [op_base.cc:1107] [619757][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4978119]us, rankNum[8], rank[3] [WARNING] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.339.317 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.339.347 [op_base.cc:853] [619757]HcclGetCommName input handle=0xfffe94022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.339.379 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 3 [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.339.399 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 3, group name: hccl_world_group [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.339.526 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.339.550 [device.cpp:183]619757 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 3. [INFO] DEVICE(619431,fffe9b7eefa0,python3.9):2025-04-03-09:35:50.339.568 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 3, group name: hccl_world_group [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.339.582 [api_impl.cc:3419] 619757 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.339.597 [runtime.cc:3444] 619757 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.339.612 [api_impl.cc:3449] 619757 DeviceReset: Succ, userDeviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.339.623 [device.cpp:189]619757 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 3. [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.339.645 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:50.339.730 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5246.51 msec. [WARNING] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:50.339.767 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.339.834 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5747.84 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.339.868 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5747.89 msec. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.339.885 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.339.908 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9879.54 msec. [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.339.936 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.129 [device.cpp:143]619431 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.163 [api_impl.cc:2999] 619431 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.179 [device.cc:306] 619431 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.196 [runtime.cc:3393] 619431 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.211 [runtime.cc:3560] 619431 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.224 [api_impl.cc:2987] 619431 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.236 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.252 [api_impl.cc:2992] 619431 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.267 [api_impl.cc:3025] 619431 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.278 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.289 [acl.cpp:121]619431 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.303 [device.cpp:154]619431 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.354.341 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.362 [context.cpp:174]619431 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.354.395 [api_impl.cc:6913] 619431 CtxSetSysParamOpt: curCtx = 0xaaaaf46ef9e0, configOpt=0, configVal=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.407 [context.cpp:158]619431 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.354.947 [op_base.cc:1214] [619431][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.354.966 [op_base.cc:128] [619431][HcclGetDeviceId] deviceLogicId[3] [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.354.987 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.355.001 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.355.015 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.049 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.485 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.770 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.818 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 119.1 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.355.968 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.007 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 35.31 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.030 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.049 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 17.6 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.065 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.105 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 37.53 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.134 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.367 msec. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.214 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.267 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 5.15 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.322 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 31.39 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.376 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.159 msec. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.403 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.439 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.485 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 43.07 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.506 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.540 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 32.26 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.561 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.581 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 18.92 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.599 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.631 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 28.9 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.669 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.262 msec. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.701 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.790 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.861 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 67.26 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.915 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 28.301 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.356.956 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 18.55 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.011 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 32.23 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 18.4 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.174 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 93.37 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.216 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.87 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.236 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.303 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 47.82 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.354 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 27.18 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.375 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.439 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 43.76 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.825 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 357.671 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.850 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.902 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 32.16 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.951 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.245 msec. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.357.971 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.016 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 1.801 msec. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.403 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 48.68 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.431 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.508 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 73.82 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.533 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.582 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 45.54 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.601 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.633 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 29.18 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.677 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.709 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 50.3 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.734 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 5.22 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.796 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.819 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 59.76 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.836 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.851 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 14.2 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.867 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.889 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 19.98 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.919 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.938 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 31.98 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.954 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.971 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 15.511 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.358.987 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.004 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 15.4 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.019 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.058 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 32.65 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.100 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 41.58 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.137 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.166 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 27.07 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.183 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.199 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 14.92 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.216 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.233 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 14.93 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.246 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.262 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 14.98 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.293 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.311 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 29.62 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.327 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 55.24 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.408 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.426 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 16.67 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.442 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.459 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 15.83 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.475 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.491 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 15.24 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.506 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.521 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 13.54 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.538 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.556 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 16.02 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.599 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.615 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 36.03 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.644 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.660 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 27.081 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.678 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.693 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 13.4 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.708 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.723 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 14.08 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.359.806 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 63.22 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.025 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 190.41 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.050 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 46.54 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.137 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.230 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 73.26 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.258 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.307 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 28.2 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.328 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.373 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 25.58 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.391 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.437 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 19.35 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.454 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.529 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 57.29 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.550 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.593 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 24.95 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.612 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.710 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 77.13 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.734 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.809 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 54.18 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.832 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.854 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.360.978 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 123.11 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.000 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.020 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 121.65 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.167 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.254 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 66.49 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.278 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.361 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 60.42 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.395 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.414 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.430 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.444 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.460 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.474 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.486 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.502 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.517 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.533 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.547 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.562 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.576 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.590 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.604 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.857 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 227.421 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.882 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.900 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.928 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.943 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.361.959 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.051 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 71.31 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.074 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.091 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.107 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.182 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 55.46 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.233 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.167 msec. [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.515 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.629 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.362.890 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 200.98 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.006 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 28.96 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.122 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 82.791 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.162 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 11.09 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.214 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 25.99 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.334 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 96.16 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.387 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 26.62 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.450 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 23.26 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.533 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 58.22 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.563 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 6.27 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.603 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.649 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.652 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.024 msec. [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.363.696 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.000 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.034 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.35 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.053 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.069 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.193 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 64.51 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.282 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 54.17 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.350 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 41.03 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.439 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 60.701 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.462 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.480 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.538 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 37.18 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.574 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 11.82 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.593 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 0.41 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.650 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.559 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.696 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.607 msec. [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.735 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.838 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.863 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 0.65 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.899 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 16.4 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.922 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 3.41 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.942 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.52 us [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.364.971 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.044 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 82.53 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.067 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 0.55 us [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.094 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.279 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.137 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.416 msec. [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.170 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.247 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.084 msec. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.365.581 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.365.611 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.365.630 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.667 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.923 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.365.966 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.059 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.158 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.107 msec. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.237 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.273 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.297 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.315 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.331 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.349 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.393 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.366.589 [context.cpp:72]619790 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.366.627 [api_impl.cc:3717] 619790 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.366.646 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.366.648 [awatchdog_core.c:175](tid:619790) create watchdog for id : 65543, tid : 619790, timeout : 300s [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.366.674 [context.cpp:80]619790 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.366.709 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.366.741 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.366.759 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.368.113 [api_impl.cc:7115] 619790 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5a00000 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.372.185 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.372.187 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.372.217 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.372.263 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.273 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.279 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.286 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.292 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.372.298 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.372.308 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.372.311 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.372.324 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.330 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.372.342 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.374.568 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.606 [api_impl.cc:7129] 619790 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.726 [api_impl.cc:7145] 619790 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.374.803 [device.cpp:143]619790 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.823 [api_impl.cc:2999] 619790 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.835 [device.cc:306] 619790 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.849 [runtime.cc:3393] 619790 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.864 [runtime.cc:3560] 619790 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.875 [api_impl.cc:2987] 619790 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.888 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.904 [api_impl.cc:2992] 619790 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.374.918 [api_impl.cc:3025] 619790 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.374.932 [acl.cpp:115]619790 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.374.953 [acl.cpp:121]619790 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.374.968 [device.cpp:154]619790 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.375.001 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.018 [context.cpp:174]619790 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.037 [api_impl.cc:6913] 619790 CtxSetSysParamOpt: curCtx = 0xaaaaf46ef9e0, configOpt=0, configVal=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.050 [context.cpp:158]619790 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.375.120 [op_base.cc:1214] [619790][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.141 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.375.154 [adapter_rts.cc:234] [619790][hrtGetDevice]deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:50.375.165 [op_base.cc:128] [619790][HcclGetDeviceId] deviceLogicId[3] [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.196 [context.cpp:72]619790 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.209 [api_impl.cc:3717] 619790 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.222 [context.cpp:80]619790 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.246 [api_error.cc:1807] 619790 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.264 [memory_task.cc:342] 619790 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.277 [memory_task.cc:473] 619790 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.288 [memory_task.cc:518] 619790 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.305 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.323 [memory_task.cc:1007] 619790 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaaaff3d6860, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.336 [memory_task.cc:1166] 619790 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.347 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.360 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.380 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.395 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.412 [callback.cpp:97]619790 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.430 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.454 [task_info.cc:521] 619790 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.468 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.479 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.493 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.505 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.515 [callback.cpp:109]619790 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.562 [npu_driver.cc:3844] 619584 CqReportGet: cq report get start, device_id=3, tsId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.578 [npu_driver.cc:3851] 619584 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.588 [api_impl.cc:5087] 619584 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.375.591 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.605 [callback.cpp:138]619584 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.375.615 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.375.620 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.375.628 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] DEVICE(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.375.637 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.376.971 [api_impl.cc:7115] 619790 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12d100000000, end ptr=0x12df35a00000 [INFO] PRE_ACT(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.383.764 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.383.799 [api_impl.cc:7129] 619790 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.383.897 [api_impl.cc:7145] 619790 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.384.016 [device.cpp:231]619790 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.034 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.384.050 [acl.cpp:802]619790 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.189 [npu_driver.cc:404] 619790 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.230 [stream.cc:2246] 619790 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.264 [stream_sqcq_manage.cc:147] 619790 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.278 [npu_driver.cc:882] 619790 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.585 [npu_driver.cc:6125] 619790 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.634 [npu_driver.cc:6137] 619790 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1052, streamId=6, logicCq=1052, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.657 [npu_driver.cc:6189] 619790 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=6, logicCq=1052. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.674 [npu_driver.cc:5916] 619790 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec5e5000, addr=dfffec5e5000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.688 [stream.cc:653] 619790 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.734 [stream.cc:276] 619790 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.748 [stream.cc:688] 619790 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=3, streamResId=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.384.763 [acl_resource_manager.cpp:814]619790 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.384.775 [acl_resource_manager.cpp:816]619790 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.911 [engine.cc:188] 619790 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.930 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.945 [task_info.cc:409] 619790 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.957 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.969 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.985 [npu_driver.cc:751] 619790 SqTaskSend: device_id=3, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.384.997 [stars_engine.cc:893] 619790 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.011 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.024 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.036 [stream.cc:2886] 619790 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.049 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.061 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.074 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.095 [logger.cc:152] 619790 TaskFinished: device_id=3, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.113 [stars_engine.cc:437] 619790 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.126 [stream.cc:2886] 619790 StarsWaitForTask: No.1 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.138 [stream.cc:852] 619790 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.151 [stream.cc:938] 619790 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.162 [stream.cc:1507] 619790 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.187 [stream_sqcq_manage.cc:451] 619790 FreeLogicCq: Return(0), threadIdentifier(281469557272480), devId(3), tsId(0), cqId(1052), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.205 [npu_driver.cc:6163] 619790 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=6, logicCq=1052. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.234 [npu_driver.cc:1180] 619790 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1052, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.247 [stream.cc:237] 619790 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.258 [stream_sqcq_manage.cc:216] 619790 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.269 [npu_driver.cc:966] 619790 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.841 [npu_driver.cc:974] 619790 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.857 [stream_sqcq_manage.cc:228] 619790 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.869 [npu_driver.cc:432] 619790 StreamIdFree: stream id free begin, device_id=3, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.885 [npu_driver.cc:441] 619790 StreamIdFree: Stream id free success, stream_id=6, device_id=3, tsId=0, streamFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.385.901 [device_msg_handler.cc:110] 619790 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.386.595 [device_msg_handler.cc:127] 619790 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.386.647 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.386.677 [npu_driver.cc:500] 619790 GetFaultEvent: get fault event, device_id=3, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.386.769 [acl.cpp:782]619790 GetFaultEventInfo: can not get fault event of device 3, runtime errorCode is 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:50.386.791 [error_manager.cc:989]619790 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.387.118 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.387.158 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.387.176 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.387.200 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.387.430 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.544 [context.cpp:72]619790 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.564 [api_impl.cc:3717] 619790 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.578 [context.cpp:80]619790 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.597 [api_error.cc:1807] 619790 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.612 [memory_task.cc:342] 619790 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.623 [memory_task.cc:473] 619790 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.387.626 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.633 [memory_task.cc:518] 619790 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.656 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.670 [memory_task.cc:1007] 619790 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaaaff3d7280, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.683 [memory_task.cc:1166] 619790 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.695 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.707 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.723 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.736 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.754 [callback.cpp:97]619790 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.772 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.787 [task_info.cc:521] 619790 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.799 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.811 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.832 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.844 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.854 [callback.cpp:109]619790 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.875 [npu_driver.cc:3844] 619584 CqReportGet: cq report get start, device_id=3, tsId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.891 [npu_driver.cc:3851] 619584 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.902 [api_impl.cc:5087] 619584 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.916 [callback.cpp:138]619584 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.926 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.926 [device.cpp:231]619790 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.938 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.942 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.387.954 [acl.cpp:802]619790 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.387.983 [npu_driver.cc:404] 619790 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.007 [stream.cc:2246] 619790 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.042 [stream_sqcq_manage.cc:147] 619790 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.057 [npu_driver.cc:882] 619790 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.250 [npu_driver.cc:6125] 619790 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.293 [npu_driver.cc:6137] 619790 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1053, streamId=7, logicCq=1053, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.312 [npu_driver.cc:6189] 619790 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=7, logicCq=1053. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.328 [npu_driver.cc:5916] 619790 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec5e5000, addr=dfffec5e5000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.340 [stream.cc:653] 619790 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.371 [stream.cc:276] 619790 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.384 [stream.cc:688] 619790 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=3, streamResId=65534 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.388.398 [acl_resource_manager.cpp:814]619790 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.388.409 [acl_resource_manager.cpp:816]619790 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.453 [engine.cc:188] 619790 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.469 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.482 [task_info.cc:409] 619790 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.493 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.505 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.520 [npu_driver.cc:751] 619790 SqTaskSend: device_id=3, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.530 [stars_engine.cc:893] 619790 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.543 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.555 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.567 [stream.cc:2886] 619790 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.578 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.589 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.603 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.615 [logger.cc:152] 619790 TaskFinished: device_id=3, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.627 [stars_engine.cc:437] 619790 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.640 [stream.cc:2886] 619790 StarsWaitForTask: No.1 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.656 [stream.cc:852] 619790 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.668 [stream.cc:938] 619790 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.679 [stream.cc:1507] 619790 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.699 [stream_sqcq_manage.cc:451] 619790 FreeLogicCq: Return(0), threadIdentifier(281469557272480), devId(3), tsId(0), cqId(1053), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.720 [npu_driver.cc:6163] 619790 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=7, logicCq=1053. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.747 [npu_driver.cc:1180] 619790 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1053, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.762 [stream.cc:237] 619790 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.776 [stream_sqcq_manage.cc:216] 619790 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.388.786 [npu_driver.cc:966] 619790 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.281 [npu_driver.cc:974] 619790 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.305 [stream_sqcq_manage.cc:228] 619790 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.323 [npu_driver.cc:432] 619790 StreamIdFree: stream id free begin, device_id=3, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.340 [npu_driver.cc:441] 619790 StreamIdFree: Stream id free success, stream_id=7, device_id=3, tsId=0, streamFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.353 [device_msg_handler.cc:110] 619790 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.580 [device_msg_handler.cc:127] 619790 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.609 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.389.628 [npu_driver.cc:500] 619790 GetFaultEvent: get fault event, device_id=3, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.389.682 [acl.cpp:782]619790 GetFaultEventInfo: can not get fault event of device 3, runtime errorCode is 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:50.389.696 [error_manager.cc:989]619790 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.389.981 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.390.010 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.390.029 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.390.043 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.390.110 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.181 [context.cpp:72]619790 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.201 [api_impl.cc:3717] 619790 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.215 [context.cpp:80]619790 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.234 [api_error.cc:1807] 619790 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.246 [memory_task.cc:342] 619790 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.257 [memory_task.cc:473] 619790 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.269 [memory_task.cc:518] 619790 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.281 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.293 [memory_task.cc:1007] 619790 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaaaff3d7cf0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.317 [memory_task.cc:1166] 619790 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.333 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.345 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.360 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.374 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.390 [callback.cpp:97]619790 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.407 [task_res.cc:380] 619790 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.425 [task_info.cc:521] 619790 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.439 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.453 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.468 [task_submit.cc:573] 619790 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.482 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.493 [callback.cpp:109]619790 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.503 [npu_driver.cc:3844] 619584 CqReportGet: cq report get start, device_id=3, tsId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.515 [npu_driver.cc:3851] 619584 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.524 [api_impl.cc:5087] 619584 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.536 [callback.cpp:138]619584 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.549 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.553 [device.cpp:231]619790 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.563 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.570 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.390.581 [acl.cpp:802]619790 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.606 [npu_driver.cc:404] 619790 StreamIdAlloc: stream id alloc begin, device_id=3, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.629 [stream.cc:2246] 619790 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.674 [stream_sqcq_manage.cc:147] 619790 Alloc: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.390.689 [npu_driver.cc:882] 619790 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.080 [npu_driver.cc:6125] 619790 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.116 [npu_driver.cc:6137] 619790 LogicCqAllocateV2: success: deviceId=3, tsId=0, cqId=1054, streamId=8, logicCq=1054, cqeDepth=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.133 [npu_driver.cc:6189] 619790 StreamBindLogicCq: success: deviceId=3, tsId=0, streamId=8, logicCq=1054. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.149 [npu_driver.cc:5916] 619790 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec5e5000, addr=dfffec5e5000size=4096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.161 [stream.cc:653] 619790 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.191 [stream.cc:276] 619790 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.207 [stream.cc:688] 619790 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=3, streamResId=65534 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.391.220 [acl_resource_manager.cpp:814]619790 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.391.232 [acl_resource_manager.cpp:816]619790 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.271 [engine.cc:188] 619790 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.286 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.299 [task_info.cc:409] 619790 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.310 [stream.cc:1400] 619790 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.322 [stream.cc:2595] 619790 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.336 [npu_driver.cc:751] 619790 SqTaskSend: device_id=3, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.346 [stars_engine.cc:893] 619790 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.358 [stream.cc:1494] 619790 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.369 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.382 [stream.cc:2886] 619790 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.396 [npu_driver.cc:5547] 619790 GetSqHead: deviceId=3, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.408 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.419 [task.cc:185] 619790 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.431 [logger.cc:152] 619790 TaskFinished: device_id=3, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.445 [stars_engine.cc:437] 619790 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.469 [stream.cc:2886] 619790 StarsWaitForTask: No.1 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.481 [stream.cc:852] 619790 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.492 [stream.cc:938] 619790 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.503 [stream.cc:1507] 619790 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.520 [stream_sqcq_manage.cc:451] 619790 FreeLogicCq: Return(0), threadIdentifier(281469557272480), devId(3), tsId(0), cqId(1054), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.538 [npu_driver.cc:6163] 619790 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=8, logicCq=1054. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.564 [npu_driver.cc:1180] 619790 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1054, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.576 [stream.cc:237] 619790 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.588 [stream_sqcq_manage.cc:216] 619790 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.391.598 [npu_driver.cc:966] 619790 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.098 [npu_driver.cc:974] 619790 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.116 [stream_sqcq_manage.cc:228] 619790 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.131 [npu_driver.cc:432] 619790 StreamIdFree: stream id free begin, device_id=3, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.149 [npu_driver.cc:441] 619790 StreamIdFree: Stream id free success, stream_id=8, device_id=3, tsId=0, streamFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.178 [device_msg_handler.cc:110] 619790 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.405 [device_msg_handler.cc:127] 619790 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.433 [runtime.cc:5239] 619790 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.392.450 [npu_driver.cc:500] 619790 GetFaultEvent: get fault event, device_id=3, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.392.499 [acl.cpp:782]619790 GetFaultEventInfo: can not get fault event of device 3, runtime errorCode is 0 [INFO] GE(619431,python3.9):2025-04-03-09:35:50.392.514 [error_manager.cc:989]619790 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.392.802 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.392.834 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.392.854 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.392.872 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619431,fffebcf9efa0,python3.9):2025-04-03-09:35:50.392.939 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.168 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.175 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.176 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.177 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.177 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.404.179 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.280 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.305 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.312 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.317 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.340 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.343 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.348 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.356 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.385 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.390 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.391 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.381 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.399 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.403 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.415 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.422 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.444 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.449 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.457 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.461 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.470 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.502 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.505 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.517 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.546 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.550 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.556 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.585 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.594 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.584 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.604 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.614 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.404.622 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.628 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.645 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.404.658 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:50.467.621 [awatchdog_monitor.c:160](tid:619467) thread 619757 not exist [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.884.190 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.884.221 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.884.235 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.884.249 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.884.277 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.884.308 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.884.328 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.916.176 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:50.916.213 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.916.228 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:50.916.247 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.916.269 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.916.286 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.916.301 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.972.792 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.972.837 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.972.859 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.972.916 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.972.937 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.972.960 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.023 [stars_engine.cc:643] 619431 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.065 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.083 [task_manager.cc:485] 619431 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.119 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.137 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.151 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.170 [stars_engine.cc:643] 619431 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.184 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.197 [task_manager.cc:485] 619431 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.225 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.240 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.254 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.272 [stars_engine.cc:643] 619431 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.288 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.302 [task_manager.cc:485] 619431 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.327 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.346 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.361 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.388 [stars_engine.cc:622] 619431 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=1048, retCode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.404 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.420 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.973.434 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.973.454 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.473 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.488 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.504 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.522 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.536 [stream.cc:2886] 619431 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.550 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.564 [stream.cc:2886] 619431 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.576 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.591 [stream.cc:2886] 619431 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.606 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.631 [stream.cc:2886] 619431 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.645 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.658 [stream.cc:2886] 619431 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.673 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.687 [stream.cc:2886] 619431 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.702 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.717 [stream.cc:2886] 619431 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.732 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.746 [stream.cc:2886] 619431 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.760 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.775 [stream.cc:2886] 619431 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.789 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.801 [stream.cc:2886] 619431 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.816 [event.cc:310] 619431 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.843 [task_res.cc:380] 619431 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=3, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.868 [event.cc:249] 619431 UpdateLatestRecord: device_id=3, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.884 [event_task.cc:259] 619431 ConstructSqeForEventRecordTask: event_record: device_id=3, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.901 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.916 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.937 [task_submit.cc:573] 619431 AllocTaskAndSendStars: device_id=3, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.953 [stars_engine.cc:1122] 619431 SyncTask: Task Wait: device_id=3, stream_id=3, task_id=0, logicCqId=1049. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.974 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.973.987 [stars_engine.cc:1071] 619431 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.003 [stars_engine.cc:1017] 619431 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.728 [stars_engine.cc:1027] 619431 ProcReport: taskType=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.751 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.774 [event.cc:249] 619431 UpdateLatestRecord: device_id=3, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.791 [event_task.cc:211] 619431 DoCompleteSuccessForEventRecordTask: event_record: device_id=3, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea5710b113 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.808 [event.cc:122] 619431 TryFreeEventIdAndCheckCanBeDelete: device_id=3, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.824 [task_res.cc:323] 619431 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=3, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.849 [npu_driver.cc:5618] 619431 GetSqEnable: deviceId=3, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.863 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.878 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.974.889 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.974.959 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.974.976 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.974.991 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.975.007 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.023 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.038 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.053 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.067 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.975.080 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.975.091 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.109 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.123 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.157 [stars_engine.cc:622] 619431 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=1049, retCode=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.171 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:50.975.185 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.975.198 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:50.975.210 [memory.cpp:421]619431 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.62s ============================================================================================================== em name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340856] [debug] [619757] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340863] [debug] [619757] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340870] [debug] [619757] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340875] [debug] [619757] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360616] [debug] [619757] [lcal_comm.cpp:254] InitCommMem 3/8, localRank_ : 3, localRankSize_ : 8 success [2025-04-03 09:35:45.360735] [info] [619757] [lcal_comm.cpp:259] LcalCommInit 3/8 success and extraFlag:0 commArgs_.localRank : 3 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.383977] [error] [619790] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.387903] [error] [619790] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.390533] [error] [619790] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.984.768 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.984.809 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:50.984.948 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619431:281473273507872,MainProcess):2025-04-03-09:35:50.985.144 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.985.223 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.985.285 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.985.374 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:50.985.396 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.232.186 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.232.306 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.232.324 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.232.342 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.232.354 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.232.367 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.232.378 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.236.176 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.236.182 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.236.183 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.236.185 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.236.186 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.281 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.302 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.306 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.316 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.326 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.331 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.336 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.343 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.351 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.366 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.378 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.382 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.391 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.409 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.415 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.403 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.429 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.437 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.451 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.444 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.463 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.452 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.473 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.508 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.534 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.510 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.236.551 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.573 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.561 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.236.589 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.396.175 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.396.207 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.396.238 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.396.253 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.396.264 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.396.276 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.396.288 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.428.184 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.428.223 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.428.239 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.428.257 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.428.273 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.428.291 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.428.308 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.464.499 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.464.636 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619431,ffff39e5dfa0,python3.9):2025-04-03-09:35:51.464.737 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.464.844 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.464.865 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619431,ffff3a66dfa0,python3.9):2025-04-03-09:35:51.464.924 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.003 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.025 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.110 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619431,ffff38e3dfa0,python3.9):2025-04-03-09:35:51.465.328 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.562 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.585 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619431,ffff3964dfa0,python3.9):2025-04-03-09:35:51.465.629 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.824 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.843 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.862 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.465.881 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.082 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.109 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.127 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.157 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.175 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.194 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.210 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.225 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.237 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.254 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.271 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.283 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.297 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.309 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.321 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.340 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.357 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.369 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.382 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.394 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.405 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.416 [stream.cpp:131]619431 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.439 [api_impl.cc:1394] 619431 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.452 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.466 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.466.478 [api_error.cc:895] 619431 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.466.490 [stream.cpp:146]619431 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.541 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.615 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.647 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.663 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.678 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.879 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.466.899 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.467.594 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.467.639 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.467.657 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.467.673 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.467.690 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.132 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.173 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.192 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.258 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.276 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.468.291 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_3 [INFO] GE_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.477.596 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.477.631 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.477.666 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.680 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.703 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.718 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.731 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.744 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.756 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.811 [device.cpp:143]619431 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.831 [api_impl.cc:2999] 619431 SetDevice: device_id=3, deviceMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.847 [device.cc:306] 619431 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.862 [runtime.cc:3393] 619431 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.877 [runtime.cc:3560] 619431 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.891 [api_impl.cc:2987] 619431 NewDevice: SetCurRef = 0xaaaaf46ef9e0,userDeviceId=3,deviceId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.905 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.943 [api_impl.cc:2992] 619431 NewDevice: New device success, user_device_id=3, die_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.477.959 [api_impl.cc:3025] 619431 SetDevice: SetDevice success, curCtx=0xaaaaf46ef9e0, user_device_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.984 [acl.cpp:115]619431 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.477.998 [acl.cpp:121]619431 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.478.022 [device.cpp:154]619431 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 3 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:51.478.046 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.532.742 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:51.532.778 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:51.532.794 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:51.532.813 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.532.834 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.532.916 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:51.622.228 [awatchdog_monitor.c:160](tid:619467) thread 619790 not exist [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.908.170 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.908.204 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.908.216 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.908.228 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.908.238 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.908.252 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.908.266 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.940.176 [trs_cb_event.c:119][ascend][curpid:619431,619584][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =64; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:51.940.204 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.940.221 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:51.940.236 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.940.250 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:51.940.264 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:51.940.278 [npu_driver.cc:3828] 619584 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=64. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.176 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.175 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.180 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.179 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.176 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.068.180 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.306 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.310 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.325 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.326 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.343 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.348 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.355 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.357 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.365 [tensor_data_transfer.cpp:1054]619615 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.368 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.377 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.386 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.394 [tensor_data_transfer.cpp:1056]619615 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.398 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.400 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.407 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.415 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.421 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.425 [api_impl.cc:6121] 619615 MemQueuePeek: Start to peek queue on devId 3, qid is 0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.438 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.442 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.450 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.457 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.466 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.475 [npu_driver.cc:4330] 619615 MemQueuePeek: peek mem queue, deviceId=3, qid=0, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.489 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.501 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.507 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.530 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.578 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.593 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.513 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.540 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.068.618 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.650 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.068.705 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] DEVICE(619431,fffe9b7eefa0,python3.9):2025-04-03-09:35:52.339.830 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.339.832 [op_base.cc:2136] [619431]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.339.874 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.339.894 [adapter_rts.cc:198] [619431][hrtGetDeviceRefresh]deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.339.945 [op_base.cc:2151] [619431][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.339.984 [op_base.cc:128] [619431][HcclGetDeviceId] deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.017 [op_base.cc:2168] [619431]com is not global com [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.038 [hcom_plugin.cc:381] [619431][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.102 [hccl_comm.cc:1375] [619431]UnRegistTaskAbortHandler begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.119 [task_abort_handler.cc:140] [619431]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.340.135 [api_impl.cc:4192] 619431 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.340.151 [runtime.cc:4698] 619431 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.183 [hccl_communicator.cc:2112] [619431][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.340.201 [hccl_communicator.cc:2114] [619431][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.215 [hccl_communicator.cc:2082] [619431]start to stop the backGround Thread [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.340.231 [heartbeat.cc:1203] [619431]Heartbeat has been destroyed [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.246 [heartbeat.cc:451] [619431][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.340.260 [heartbeat.cc:452] [619431]Heartbeat has been destroyed [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.314 [hccl_impl.cc:69] [619431]start hccl resources destruction:deviceLogicId[3] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.340.349 [heartbeat.cc:1203] [619431]Heartbeat has been destroyed [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.340.374 [heartbeat.cc:451] [619431][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.340.388 [heartbeat.cc:452] [619431]Heartbeat has been destroyed [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.340.729 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.341.099 [task_exception_handler.cc:1232] [619431]deInit taskFailCallback [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.341.810 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.341.833 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.341.935 [api_error.cc:1278] 619431 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.341.951 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.341.984 [hccl_socket_manager.cc:81] [619431][DeInit][Server]ip[192.168.2.196] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.002 [hccl_socket_manager.cc:83] [619431][DeInit][Server]ip[192.168.2.196] port[16666] [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.342.015 [hccl_socket.cc:232] [619431][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.029 [network_manager.cc:69] [619431]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.044 [network_manager.cc:458] [619431]NetworkManager devicePhyId_[3] vnic stopped. refcount[0] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:52.342.065 [ra_host.c:911]tid:619431,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[3], local_ip[192.168.2.196] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:52.342.394 [ra_host.c:404]tid:619431,ra_socket_deinit(404) : Input parameters: phy_id[3] family[2] local_ip[192.168.2.196] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.539 [hccl_socket.cc:68] [619431][HcclSocket][DeInit] device[3] stops listen on ip[192.168.2.196], port[16666] success, socketType[0]. [WARNING] HCCL(619431,python3.9):2025-04-03-09:35:52.342.560 [hccl_socket.cc:232] [619431][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.585 [hccl_communicator.cc:1741] [619431][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.602 [network_manager.cc:69] [619431]NetworkManager::GetInstance deviceLogicID[3]. [INFO] HCCP(619431,python3.9):2025-04-03-09:35:52.342.617 [ra_host.c:248]tid:619431,ra_deinit(248) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(619431,python3.9):2025-04-03-09:35:52.342.630 [ra_hdc.c:651]tid:619431,ra_hdc_deinit(651) : hdc deinit start! phy_id[3] [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.342.727 [hdc_client.c:417][ascend][curpid:619431,619431][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4273) [INFO] ROCE(619431,python3.9):2025-04-03-09:35:52.342.794 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619431,python3.9):2025-04-03-09:35:52.342.809 [ra_hdc.c:685]tid:619431,ra_hdc_deinit(685) : hdc deinit OK! phy_id[3] [INFO] ROCE(619431,python3.9):2025-04-03-09:35:52.342.824 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.881 [tsd_client.cpp:189][TsdCapabilityGet][tid:619431] TsdCapabilityGet Begin. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.895 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.919 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.932 [process_mode_manager.cpp:1042][CapabilityGet][tid:619431] [ProcessModeManager] enter into CapabilityGet process deviceId[3] type[6] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.342.947 [adapter_tdt.cc:66] [619431]Get TsdCapability success. deviceLogicId[3] [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.961 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.973 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.342.984 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619431] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.343.000 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619431] add close subproc:31203, proctype:0 [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.343.033 [hdc_client.cpp:154][TsdRecvData][tid:619431] recv timeout is:150000 ms [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.414.904 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619431] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.414.924 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619431] [TsdClient] DeviceMsgProc recvMsg realDeviceId[3] msgType[61] localDevId[3] rspCode[0]heterogeneousSubPid[31203], tsdSupportLevel_[63] [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.414.943 [hdc_client.cpp:172][TsdRecvData][tid:619431] [deviceId=3] the Tsd recv data already [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.414.959 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619431] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.414.974 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619431] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.414.987 [adapter_tdt.cc:52] [619431]Open TsdProcess success. deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.000 [network_manager.cc:405] [619431][DeInit]finish CloseHccpProcess, phy_id[3], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.014 [network_manager.cc:409] [619431]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.028 [hccl_communicator_attrs.cc:1307] [619431][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.042 [hccl_communicator_attrs.cc:1275] [619431]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.061 [notify_pool_impl.cc:43] [619431]NotifyPoolImpl Destroy. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.083 [notify_pool_impl.cc:56] [619431]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.124 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.139 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.153 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.172 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.184 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.197 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.220 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.236 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.260 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.274 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.291 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=0. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.313 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=0.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.330 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.344 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.355 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.366 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.380 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.394 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.412 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.424 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=1. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.440 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=1.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.453 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.465 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.477 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.489 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.502 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.516 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.529 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.540 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=2. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.555 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=2.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.568 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.581 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.595 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.608 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.621 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.631 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.656 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.668 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.679 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.694 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.707 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.720 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=4. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.734 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=4.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.748 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.759 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.770 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.782 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.793 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.806 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.820 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.832 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=5. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.846 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=5.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.859 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.871 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.882 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.896 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.908 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.919 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.415.932 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.945 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=6. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.415.960 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=6.) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.973 [api_impl.cc:3100] 619431 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.985 [api_impl.cc:3111] 619431 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.415.995 [api_impl.cc:3130] 619431 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.416.017 [p2p_mgmt.cc:171] [619431]local logic id:3, local physic id:3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.030 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.047 [api_impl.cc:3520] 619431 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.416.061 [p2p_mgmt.cc:178] [619431]disable p2p: local logic id:3, remote physic id:7. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.073 [api_impl.cc:3184] 619431 DisableP2P: Disable P2P devIdDes=3, phyIdSrc=7. [INFO] DRV(619431,python3.9):2025-04-03-09:35:52.416.087 [devdrv_pcie.c:890][ascend][curpid:619431,619431][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=3; peer_phy_devid=7.) [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.416.101 [hccl_communicator.cc:2043] [619431]Profiling is deinitiated. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.114 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.135 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.416.151 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.416.181 [opexecounter.cc:103] [619431]free counter mem resource [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.416.201 [atrace_api.c:118](tid:619431) AtraceDestroy start. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.416.217 [tracer_mgr_operate.c:248](tid:619431) destroy object HCCL_619757_3, exitSave(true). [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.416.230 [atrace_api.c:120](tid:619431) AtraceDestroy end. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.416.243 [adapter_trace.cc:33] [619431]Call TraceDestroy, Params:handle[0xaaaae2c52708] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.418.749 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.418.807 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.418.863 [queue_notify_manager.cc:100] [619431]QueueNotifyManager Destroy. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.418.879 [queue_notify_manager.cc:106] [619431]QueueNotifyManager Destroy success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.176 [queue_notify_manager.cc:100] [619431]QueueNotifyManager Destroy. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.193 [queue_notify_manager.cc:106] [619431]QueueNotifyManager Destroy success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.329 [ccl_buffer_manager.cc:125] [619431][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.348 [ccl_buffer_manager.cc:147] [619431][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.362 [ccl_buffer_manager.cc:160] [619431][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.375 [ccl_buffer_manager.cc:163] [619431][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.387 [ccl_buffer_manager.cc:166] [619431][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.449 [op_base.cc:128] [619431][HcclGetDeviceId] deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.470 [op_base.cc:2196] [619431]op_base comm destroy complete, take time [79595]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[3]. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.419.487 [device.cpp:183]619431 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.503 [api_impl.cc:3419] 619431 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.527 [runtime.cc:3444] 619431 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.541 [api_impl.cc:3449] 619431 DeviceReset: Succ, userDeviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.419.552 [device.cpp:189]619431 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 3. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.419.594 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.419.621 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.419.634 [hcom_executor.cc:40] [619431]Hcom Excutor Finalize start. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.754 [api_impl.cc:3419] 619627 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.781 [runtime.cc:3444] 619627 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.795 [api_impl.cc:3449] 619627 DeviceReset: Succ, userDeviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.819 [api_impl.cc:3419] 619629 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.845 [runtime.cc:3444] 619629 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.419.862 [api_impl.cc:3449] 619629 DeviceReset: Succ, userDeviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.056 [api_impl.cc:3419] 619630 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.077 [api_impl.cc:3419] 619628 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.085 [runtime.cc:3444] 619630 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.097 [runtime.cc:3444] 619628 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.100 [api_impl.cc:3449] 619630 DeviceReset: Succ, userDeviceId=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.111 [api_impl.cc:3449] 619628 DeviceReset: Succ, userDeviceId=3. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.420.178 [trs_cb_event.c:119][ascend][curpid:619431,619585][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=3, tid =65; ret=16) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.420.202 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.215 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.231 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.420.246 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.420.258 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.271 [npu_driver.cc:3828] 619585 CqReportIrqWait: Irq wait start, deviceId=3, tsId=0, groupId=65. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.299 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.420.320 [adapter_rts.cc:321] [619431][hrtCtxSetCurrent] success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.420.346 [stream.cc:182] [619431]Switch Ctx ret[0], curCtx[0xaaaaf46ef9e0], setCtx[0xaaaaf46ef9e0], stream id[5] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.386 [api_impl.cc:1228] 619431 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.420.404 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.420.424 [single_op_manager.cc:50]619431 ReleaseResource:ReleaseResource in. resource id = 0xaaaafec53950 [INFO] GE(619431,python3.9):2025-04-03-09:35:52.420.460 [caching_allocator.cc:402]619431 TryFreeBlocks:Try free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.420.476 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:52.420.519 [npu_memory_allocator.cc:72]619431 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.420.532 [stream_executor.cpp:197]619431 RemoveExecutor: To remove executor by stream = 187651395500368 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.420.561 [acl_resource_manager.cpp:823]619431 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.576 [stream.cc:852] 619431 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.591 [stream.cc:938] 619431 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.602 [stream.cc:1507] 619431 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.657 [stream_sqcq_manage.cc:451] 619431 FreeLogicCq: Return(0), threadIdentifier(281473273507872), devId(3), tsId(0), cqId(1051), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.684 [npu_driver.cc:6163] 619431 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=5, logicCq=1051. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.420.739 [npu_driver.cc:1180] 619431 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1051, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.423.342 [stream.cc:237] 619431 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.423.366 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.423.381 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.423.995 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.424.015 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.424.032 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.424.051 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=5, device_id=3, tsId=0, streamFlag=1536 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.425.673 [stream.cc:185] [619431][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaafec53950], stream id[5] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.697 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.425.713 [adapter_rts.cc:321] [619431][hrtCtxSetCurrent] success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.425.725 [stream.cc:193] [619431]Restore Ctx ret[0], setCtx[0xaaaaf46ef9e0], stream id[5] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.799 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.425.830 [adapter_rts.cc:321] [619431][hrtCtxSetCurrent] success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.425.842 [stream.cc:182] [619431]Switch Ctx ret[0], curCtx[0xaaaaf46ef9e0], setCtx[0xaaaaf46ef9e0], stream id[4] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.857 [api_impl.cc:1228] 619431 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.425.871 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.425.887 [single_op_manager.cc:50]619431 ReleaseResource:ReleaseResource in. resource id = 0xaaaafebd78f0 [INFO] GE(619431,python3.9):2025-04-03-09:35:52.425.902 [caching_allocator.cc:402]619431 TryFreeBlocks:Try free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.425.912 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:52.425.926 [npu_memory_allocator.cc:72]619431 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.425.937 [stream_executor.cpp:197]619431 RemoveExecutor: To remove executor by stream = 187651394992368 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.425.949 [acl_resource_manager.cpp:823]619431 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.962 [stream.cc:852] 619431 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.977 [stream.cc:938] 619431 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.425.990 [stream.cc:1507] 619431 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.426.013 [stream_sqcq_manage.cc:451] 619431 FreeLogicCq: Return(0), threadIdentifier(281473273507872), devId(3), tsId(0), cqId(1050), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.426.033 [npu_driver.cc:6163] 619431 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=4, logicCq=1050. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.426.068 [npu_driver.cc:1180] 619431 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1050, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.415 [stream.cc:237] 619431 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.437 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.451 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.908 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.926 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.942 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.428.960 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=4, device_id=3, tsId=0, streamFlag=1536 [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.382 [stream.cc:185] [619431][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaafebd78f0], stream id[4] [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.430.404 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.421 [adapter_rts.cc:321] [619431][hrtCtxSetCurrent] success [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.437 [stream.cc:193] [619431]Restore Ctx ret[0], setCtx[0xaaaaf46ef9e0], stream id[4] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.482 [hcom_executor.cc:45] [619431][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.430.519 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.430.546 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.560 [hcom_ops_kernel_info_store.cc:3423] [619431]finalize hccl kernel info store. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.580 [hcom_common.cc:85] [619431][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[3] [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.599 [plugin_manager.cc:56] [619431]hccl ops plugin finalize start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.614 [hcom_plugin.cc:90] [619431]Finalize start. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.727 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.743 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.758 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.771 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.800 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.813 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.825 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.840 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.852 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.864 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.875 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.885 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.895 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.906 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.917 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.929 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.953 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.965 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.975 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.430.987 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.015 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.027 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.038 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.050 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.077 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.099 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.113 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.125 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.154 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.166 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.177 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.189 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.216 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.229 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.240 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.251 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.275 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.286 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.297 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.308 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.335 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.348 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.359 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.370 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.395 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.409 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.422 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.434 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.463 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.476 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.487 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.500 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.526 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.539 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.550 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.565 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.593 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.607 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.626 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.639 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.665 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.677 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.689 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.700 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.711 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.722 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.734 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.746 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.759 [hcom_plugin.cc:119] [619431]Finalize: HcomDestroy success. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.770 [plugin_manager.cc:62] [619431]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.431.840 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.431.860 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.871 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.888 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.902 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.912 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.923 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.934 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.946 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.957 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.967 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.979 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.989 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.431.999 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.010 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.022 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.033 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.045 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.057 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.068 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.090 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.102 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.114 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.128 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.140 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.151 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.188 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.200 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.211 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.224 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.236 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.249 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.260 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.271 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.284 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.295 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.305 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.316 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.326 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.337 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.347 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.360 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.370 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.383 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.395 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.406 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.417 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.431 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.442 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.455 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.465 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.477 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.487 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.498 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.518 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.531 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.542 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.552 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.564 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.575 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.586 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.598 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.610 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.621 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.631 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.642 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.653 [hcom_common.cc:958] [619431][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.664 [hcom_common.cc:961] [619431]Entry-HcomDestroy:void skip [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.676 [hcom.cc:337] [619431][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:52.432.687 [hcom.cc:340] [619431]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.726 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.747 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.943 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.964 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.980 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.432.996 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.010 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.026 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.041 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.059 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.083 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.098 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.113 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.126 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.139 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.153 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.168 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.182 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.199 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.226 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.244 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.264 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.283 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.297 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.315 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.333 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.356 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.372 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.388 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.403 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.420 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.440 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.454 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.469 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.488 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe94100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.506 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe94101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.433.528 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.551 [callback.cpp:37]619431 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.572 [callback.cpp:44]619431 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.607 [context.cpp:72]619431 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.622 [api_impl.cc:3717] 619431 ContextSetCurrent: set context=aaaaf46ef9e0, flag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.636 [context.cpp:80]619431 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.651 [stream.cpp:75]619431 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619431,python3.9):2025-04-03-09:35:52.433.669 [stream_manage_func_registry.cc:22]619431 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.682 [stream.cpp:80]619431 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.698 [api_impl.cc:1228] 619431 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.721 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.433.736 [single_op_manager.cc:50]619431 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf481af80 [INFO] GE(619431,python3.9):2025-04-03-09:35:52.433.749 [caching_allocator.cc:402]619431 TryFreeBlocks:Try free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.433.762 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:52.433.773 [npu_memory_allocator.cc:72]619431 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.784 [stream_executor.cpp:197]619431 RemoveExecutor: To remove executor by stream = 187651223302016 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.433.796 [acl_resource_manager.cpp:823]619431 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.810 [stream.cc:852] 619431 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.824 [stream.cc:938] 619431 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.842 [event.cc:1036] 619431 WaitForBusy: device_id=3, event_id=0 finished map already empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.858 [npu_driver.cc:626] 619431 EventIdFree: event id free begin, device_id=3, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.885 [raw_device.cc:1158] 619431 FreeEventIdFromDrv: device_id=3, event_id=0 tear down. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.900 [event.cc:61] 619431 ~Event: device_id=3, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.925 [npu_driver.cc:3803] 619431 SqCqFree: deviceId=3, tsId=0, sqId=789, cqId=789 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.963 [subscribe.cc:244] 619431 Delete: delete callback subscribe: groupId=64, threadId=281471394574240, devId=3, tsId=0, streamId=2, sqId=789, cqId=789 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.433.970 [trs_sqcq.c:1491][ascend][curpid:619431,619584][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.980 [stream.cc:1507] 619431 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.433.996 [npu_driver.cc:3833] 619584 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.016 [api_impl.cc:5058] 619584 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.029 [stream_sqcq_manage.cc:451] 619431 FreeLogicCq: Return(0), threadIdentifier(281473273507872), devId(3), tsId(0), cqId(1048), isFastCq(0). [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.033 [callback.cpp:132]619584 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.049 [npu_driver.cc:6163] 619431 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=2, logicCq=1048. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.053 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.077 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.085 [npu_driver.cc:1180] 619431 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1048, drvFlag=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.091 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.130 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.146 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.160 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.173 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.185 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.197 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.208 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.220 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.231 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.243 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.255 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.264 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.279 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.291 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.302 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.312 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.326 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.339 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.350 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.360 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.372 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.385 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.397 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.407 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.425 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.437 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.448 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.461 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.473 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.483 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.496 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.507 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.518 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.529 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.542 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.555 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.568 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.582 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.594 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.606 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.619 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.629 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.640 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.651 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.664 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.675 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.688 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.700 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.711 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.732 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.747 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.761 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.777 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.789 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.802 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.815 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.828 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.839 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.851 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.863 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.877 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.887 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.899 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.912 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.926 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.938 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.949 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.959 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.971 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.434.983 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.434.995 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.007 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.019 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.031 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.055 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.066 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.078 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.091 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.105 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.117 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.128 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.142 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.156 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.168 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.184 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.198 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.209 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.225 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.236 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.249 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.263 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.278 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.289 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.301 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.313 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.324 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.334 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.345 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.356 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.374 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.387 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.399 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.411 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.423 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.432 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.442 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.454 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.467 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.479 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.492 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.506 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.517 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.530 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.542 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.557 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.569 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.583 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.594 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.605 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.618 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.629 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.641 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.653 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.681 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.696 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.705 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.717 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.728 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.739 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.750 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.763 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.778 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.793 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.803 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.818 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.831 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.844 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.855 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.868 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.879 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.890 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.901 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.913 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.925 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.937 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.949 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.960 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.435.970 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.435.989 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.001 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.016 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.029 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.040 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.054 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.067 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.080 [stream.cc:237] 619431 FreeStreamId: Free stream_id=2. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.081 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.096 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.104 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.113 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=3, tsId=0, drvFlag=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.122 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.137 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.151 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.179 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.196 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.213 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.223 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.236 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.250 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.262 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.275 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.286 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.296 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.316 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.330 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.343 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.353 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.365 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.378 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.392 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.404 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.418 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.430 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.443 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.457 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.473 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.488 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.501 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.514 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.526 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.536 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.547 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.559 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.573 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.584 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.587 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=3, tsId=0, drvFlag=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.597 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.607 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.614 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.636 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=2, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.645 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.656 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=2, device_id=3, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.660 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.680 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.692 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.704 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.718 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.731 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.744 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.758 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.770 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.781 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.794 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.805 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.815 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.828 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.838 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.851 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.866 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.878 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.888 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.900 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.911 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.936 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.949 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.965 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.436.976 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.988 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.436.999 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.013 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.025 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.037 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.049 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.062 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.074 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.085 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.096 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.107 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.120 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.132 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.147 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.160 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.173 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.185 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.197 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.208 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.219 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.240 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.256 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.269 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.280 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.292 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.305 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.318 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.330 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.342 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.355 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.367 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.380 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.391 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.402 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.415 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.427 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.439 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.450 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.461 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.472 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.488 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.501 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.513 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.525 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.538 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.559 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.572 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.583 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.594 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.605 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.616 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.627 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.640 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.652 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.664 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.675 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.688 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.703 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.716 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.728 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.740 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.753 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.765 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.777 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.788 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.800 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.814 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.828 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.842 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.855 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.874 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.889 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.900 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.911 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.925 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.938 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.949 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.962 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.976 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.437.987 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.437.999 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.012 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.026 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.037 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.050 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.063 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.076 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.090 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.103 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.116 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.127 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.139 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.150 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.159 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.172 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.192 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.204 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.217 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.231 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.245 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.260 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.275 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.287 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.297 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.310 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.322 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.338 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.352 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.365 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.379 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.393 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.404 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.416 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.426 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.439 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.450 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.463 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.474 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.485 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.503 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.519 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.535 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.549 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.563 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.576 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.587 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.600 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.612 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.624 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.638 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.652 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.664 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.675 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.689 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.702 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.714 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.726 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.737 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.749 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.762 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.775 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.784 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.798 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.809 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.831 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.842 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.854 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.867 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.878 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.890 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.904 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.915 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.927 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.939 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.951 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.961 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.974 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.438.986 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.438.998 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.009 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.020 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.031 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.043 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.054 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.066 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.080 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.095 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.108 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.120 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.149 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.165 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.180 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.193 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.207 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.221 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.231 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.242 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.255 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.266 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.279 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.292 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.306 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.322 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.337 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.349 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.363 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.379 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.392 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.408 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.424 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.437 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.448 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.461 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.476 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.496 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.508 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.519 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.532 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.545 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.557 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.569 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.581 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.596 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.607 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.620 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.632 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.644 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.659 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.675 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.688 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.703 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.716 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.731 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.744 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.758 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.773 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.788 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.802 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.813 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.833 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.844 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.854 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.867 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.879 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.891 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.902 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.916 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.929 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.945 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.439.960 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.976 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.439.991 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.007 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.020 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.032 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.044 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.057 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.067 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.080 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.091 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.105 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.118 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.132 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.152 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.181 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.193 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.207 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.219 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.234 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.246 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.259 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.270 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.281 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.292 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.303 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.313 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.327 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.338 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.353 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.366 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.378 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.392 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.405 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.419 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.433 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.446 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.457 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.468 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.490 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.503 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.517 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.531 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.545 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.558 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.570 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.581 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.591 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.605 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.619 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.633 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.645 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.659 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.673 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.688 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.704 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.718 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.732 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.743 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.757 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.770 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.783 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.794 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.806 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.826 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.841 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.855 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.869 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.881 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.894 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.906 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.917 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.930 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.943 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.954 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.966 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.440.976 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.440.987 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.001 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.015 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.026 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.039 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.049 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.062 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.072 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.085 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.096 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.108 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.119 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.139 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.152 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.164 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.176 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.189 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.201 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.215 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.227 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.239 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.248 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.259 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.273 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.285 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.298 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.310 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.321 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.333 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.346 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.358 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.370 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.385 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.398 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.411 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.424 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.439 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.458 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.472 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.485 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.498 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.511 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.522 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.533 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.545 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.558 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.572 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.582 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.593 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.604 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.614 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.627 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.639 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.651 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.665 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.676 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.689 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.702 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.715 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.727 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.739 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.759 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.775 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.786 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.800 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.813 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.826 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.838 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.849 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.860 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.871 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.883 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.896 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.908 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.919 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.931 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.943 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.955 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.969 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.441.980 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.441.993 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.006 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.019 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.032 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.043 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.057 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.080 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.092 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.106 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.119 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.131 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.144 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.155 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.166 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.179 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.190 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.201 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.212 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.223 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.234 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.246 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.256 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.267 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.278 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.292 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.304 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.315 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.328 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.341 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.353 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.365 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.386 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.400 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.413 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.426 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.437 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.449 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.461 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.472 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.483 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.495 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.509 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.522 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.533 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.546 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.559 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.571 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.585 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.597 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.610 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.621 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.634 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.646 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.658 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.669 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.680 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.702 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.715 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.730 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.743 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.759 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.772 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.784 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.795 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.811 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.824 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.834 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.847 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.862 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.874 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.885 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.895 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.907 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.917 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.930 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.941 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.954 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.442.966 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.977 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.442.988 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.010 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.025 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.038 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.053 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.066 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.078 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.089 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.101 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.112 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.125 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.136 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.150 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.162 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.173 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.186 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.199 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.210 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.221 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.233 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.248 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.262 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.274 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.286 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.297 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.308 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.327 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.338 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.351 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.364 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.377 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.390 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.402 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.416 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.429 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.443 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.456 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.470 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.482 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.493 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.504 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.517 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.530 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.544 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.556 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.570 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.582 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.593 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.605 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.617 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.629 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.652 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.665 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.680 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.692 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.705 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.718 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.730 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.740 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.752 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.766 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.780 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.791 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.802 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.815 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.828 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.842 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.853 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.864 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.877 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.887 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.898 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.909 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.922 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.934 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.947 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.443.967 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.978 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.443.988 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.001 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.013 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.024 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.036 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.050 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.065 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.078 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.091 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.103 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.117 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.131 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.144 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.166 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.177 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.190 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.202 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.215 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.225 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.237 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.251 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.263 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.275 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.294 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.307 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.319 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.331 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.342 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.353 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.365 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.378 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.391 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.403 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.414 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.427 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.438 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.448 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.459 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.473 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.484 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.495 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.505 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.516 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.527 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.539 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.551 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.563 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.583 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.594 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.605 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.619 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.630 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.640 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.652 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.662 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.674 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.686 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.700 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.712 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.725 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.736 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.750 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.761 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.773 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.783 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.794 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.805 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.818 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.831 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.844 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.856 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.869 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.898 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.910 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.922 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.932 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.944 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.956 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.444.967 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.977 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.444.990 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.001 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.014 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.027 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.039 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.052 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.062 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.074 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.086 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.097 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.108 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.121 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.133 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.146 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.156 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.168 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.179 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.200 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.215 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.228 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.240 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.253 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.266 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.278 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.291 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.302 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.313 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.323 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.336 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.351 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.365 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.379 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.391 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.402 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.413 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.424 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.435 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.446 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.458 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.471 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.485 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.499 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.516 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.529 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.540 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.553 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.564 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.575 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.588 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.602 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.614 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.629 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.643 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.657 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.669 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.681 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.693 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.704 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.715 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.725 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.737 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.748 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.759 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.772 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.786 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.798 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.809 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.833 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.846 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.858 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.869 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.881 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.891 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.902 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.913 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.926 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.937 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.949 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.961 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.974 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.445.986 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.445.998 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.011 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.025 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.037 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.049 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.059 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.069 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.080 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.092 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.102 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.119 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.131 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.141 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.152 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.163 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.173 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.186 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.199 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.210 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.223 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.237 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.248 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.259 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.270 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.282 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.295 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.307 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.318 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.329 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.339 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.351 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.361 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.372 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.383 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.395 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.414 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.426 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.437 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.448 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.458 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.469 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.480 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.494 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.504 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.515 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.528 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.542 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.554 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.566 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.578 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.590 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.601 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.613 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.626 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.639 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.651 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.664 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.676 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.687 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.698 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.717 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.728 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.739 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.750 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.760 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.771 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.783 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.793 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.803 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.815 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.826 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.836 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.848 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.859 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.870 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.881 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.892 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.904 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.915 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.927 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.939 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.950 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.962 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.446.973 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.446.986 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.004 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.015 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.026 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.038 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.048 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.058 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.069 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.081 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.093 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.103 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.116 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.128 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.139 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.151 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.162 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.172 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.183 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.194 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.204 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.215 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.228 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.241 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.251 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.263 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.282 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.294 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.305 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.316 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.328 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.341 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.351 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.362 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.372 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.384 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.394 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.405 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.418 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.429 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.440 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.450 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.463 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.474 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.485 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.496 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.509 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.523 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.534 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.546 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.556 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.574 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.586 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.597 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.607 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.618 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.629 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.639 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.651 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.661 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.672 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.683 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.693 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.705 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.716 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.727 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.737 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.749 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.760 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.771 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.782 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.794 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.804 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.815 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.825 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.838 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.855 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.866 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.876 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.889 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.902 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.917 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.928 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.941 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.952 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.964 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.447.977 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.447.990 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.005 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.019 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.030 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.043 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.057 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.067 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.078 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.090 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.101 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.111 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.122 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.135 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.146 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.171 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.191 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.206 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.216 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.228 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.239 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.252 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.262 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.274 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.287 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.297 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.307 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.318 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.328 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.340 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.333 [stream.cpp:87]619431 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.352 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.371 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.383 [callback.cpp:116]619584 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.396 [subscribe.cc:308] 619584 GetGroupIdByThreadId: can not find subscribe info, threadId=281471394574240 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.407 [api_impl.cc:5043] 619584 ProcessReport: get groupId fail, threadIdentifier=281471394574240, reCode=0x7160004 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.448.402 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaaf481af80. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.419 [callback.cpp:130]619584 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.431 [stream.cpp:75]619431 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619431,python3.9):2025-04-03-09:35:52.448.452 [stream_manage_func_registry.cc:22]619431 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.474 [stream.cpp:80]619431 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] DEVICE(619431,ffff2a7cefa0,python3.9):2025-04-03-09:35:52.448.485 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.490 [api_impl.cc:1228] 619431 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.517 [acl_resource_manager.cpp:814]619431 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.448.531 [single_op_manager.cc:50]619431 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf494a360 [INFO] GE(619431,python3.9):2025-04-03-09:35:52.448.546 [caching_allocator.cc:402]619431 TryFreeBlocks:Try free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:52.448.558 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:52.448.570 [npu_memory_allocator.cc:72]619431 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.583 [stream_executor.cpp:197]619431 RemoveExecutor: To remove executor by stream = 187651224544096 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.596 [acl_resource_manager.cpp:823]619431 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.610 [stream.cc:852] 619431 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.624 [stream.cc:938] 619431 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.638 [event.cc:1036] 619431 WaitForBusy: device_id=3, event_id=1 finished map already empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.655 [npu_driver.cc:626] 619431 EventIdFree: event id free begin, device_id=3, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.679 [raw_device.cc:1158] 619431 FreeEventIdFromDrv: device_id=3, event_id=1 tear down. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.694 [event.cc:61] 619431 ~Event: device_id=3, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.712 [npu_driver.cc:3803] 619431 SqCqFree: deviceId=3, tsId=0, sqId=790, cqId=790 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.743 [subscribe.cc:244] 619431 Delete: delete callback subscribe: groupId=65, threadId=281471386120096, devId=3, tsId=0, streamId=3, sqId=790, cqId=790 [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.448.749 [trs_sqcq.c:1491][ascend][curpid:619431,619585][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=3; type=1; ret=16) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.760 [stream.cc:1507] 619431 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.769 [npu_driver.cc:3833] 619585 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=3, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.784 [api_impl.cc:5058] 619585 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.793 [stream_sqcq_manage.cc:451] 619431 FreeLogicCq: Return(0), threadIdentifier(281473273507872), devId(3), tsId(0), cqId(1049), isFastCq(0). [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.796 [callback.cpp:132]619585 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.813 [npu_driver.cc:6163] 619431 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=3, logicCq=1049. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.816 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.844 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.855 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.858 [npu_driver.cc:1180] 619431 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1049, drvFlag=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.869 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.884 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.895 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.906 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.916 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.928 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.939 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.949 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.959 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.448.971 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.981 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.448.993 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.004 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.014 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.025 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.036 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.046 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.057 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.069 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.079 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.090 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.119 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.131 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.143 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.154 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.163 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.175 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.185 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.197 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.207 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.217 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.228 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.239 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.250 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.263 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.272 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.282 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.293 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.304 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.314 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.324 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.335 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.346 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.356 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.367 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.378 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.396 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.407 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.418 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.429 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.439 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.449 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.462 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.474 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.486 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.496 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.507 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.517 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.528 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.539 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.550 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.560 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.570 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.581 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.593 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.603 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.614 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.624 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.636 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.646 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.656 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.672 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.683 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.693 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.705 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.715 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.725 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.736 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.747 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.757 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.768 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.780 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.790 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.801 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.814 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.825 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.837 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.847 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.858 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.870 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.881 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.892 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.904 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.915 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.925 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.942 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.953 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.964 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.976 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.449.986 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.449.997 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.008 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.019 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.029 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.039 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.049 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.060 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.071 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.081 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.092 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.104 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.114 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.124 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.135 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.146 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.157 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.166 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.176 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.188 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.198 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.215 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.227 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.237 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.248 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.250 [stream.cc:237] 619431 FreeStreamId: Free stream_id=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.259 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.269 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.274 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.282 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=3, tsId=0, drvFlag=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.288 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.302 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.313 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.324 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.336 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.346 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.356 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.367 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.378 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.388 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.399 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.410 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.420 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.433 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.444 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.455 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.476 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.487 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.498 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.509 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.520 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.530 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.541 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.553 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.563 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.574 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.585 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.596 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.606 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.617 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.628 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.639 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.650 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.660 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.669 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.679 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.691 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.702 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.713 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.725 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.725 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=3, tsId=0, drvFlag=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.737 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.754 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.761 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.771 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=3, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.777 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.788 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=3, device_id=3, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.792 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.810 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.820 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.829 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.838 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.848 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.859 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.869 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.880 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.891 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.901 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.912 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.922 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.933 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.943 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.954 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.450.964 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.975 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.450.990 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.001 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.012 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.023 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.033 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.044 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.054 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.065 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.074 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.084 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.092 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.103 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.113 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.124 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.134 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.145 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.156 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.166 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.177 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.187 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.198 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.208 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.219 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.229 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.240 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.257 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.268 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.279 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.289 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.300 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.310 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.321 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.331 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.340 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.352 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.363 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.374 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.384 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.395 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.406 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.416 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.427 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.437 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.448 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.451 [event.cc:61] 619431 ~Event: device_id=3, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.458 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.472 [stream.cpp:87]619431 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.472 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.494 [callback.cpp:116]619585 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.505 [subscribe.cc:308] 619585 GetGroupIdByThreadId: can not find subscribe info, threadId=281471386120096 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.508 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaaf494a360. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.519 [api_impl.cc:5043] 619585 ProcessReport: get groupId fail, threadIdentifier=281471386120096, reCode=0x7160004 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.533 [callback.cpp:130]619585 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.532 [device.cpp:183]619431 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.554 [api_impl.cc:3419] 619431 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.568 [runtime.cc:3444] 619431 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.451.582 [api_impl.cc:3449] 619431 DeviceReset: Succ, userDeviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.595 [device.cpp:189]619431 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 3. [INFO] DEVICE(619431,ffff29fbefa0,python3.9):2025-04-03-09:35:52.451.593 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.618 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.665 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.690 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.708 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.723 [callback.cpp:37]619431 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.451.740 [callback.cpp:44]619431 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.761 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.451.783 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.582.012 [awatchdog_monitor.c:160](tid:619467) thread 619627 not exist [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.582.039 [awatchdog_monitor.c:160](tid:619467) thread 619628 not exist [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.582.059 [awatchdog_monitor.c:160](tid:619467) thread 619629 not exist [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:52.582.079 [awatchdog_monitor.c:160](tid:619467) thread 619630 not exist [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.748.720 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.748.765 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.748.781 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:52.748.795 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.748.814 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.748.883 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.896.186 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.896.333 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.896.356 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.896.375 [tensor_data_transfer.cpp:1054]619625 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_histogram_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.896.392 [tensor_data_transfer.cpp:1056]619625 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.896.407 [api_impl.cc:6121] 619625 MemQueuePeek: Start to peek queue on devId 3, qid is 5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.896.422 [npu_driver.cc:4330] 619625 MemQueuePeek: peek mem queue, deviceId=3, qid=5, timeout=800. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.900.176 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.900.177 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.900.176 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.900.177 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:52.900.185 [queue_client_comm.c:114][ascend][curpid:619431,619615][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.272 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.295 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.301 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.307 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.321 [tensor_data_transfer.cpp:1054]619619 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.316 [tensor_data_transfer.cpp:1054]619623 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_scalar_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.336 [tensor_data_transfer.cpp:1056]619619 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.338 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.372 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.379 [api_impl.cc:6121] 619619 MemQueuePeek: Start to peek queue on devId 3, qid is 2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.386 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.373 [tensor_data_transfer.cpp:1056]619623 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.396 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.398 [npu_driver.cc:4338] 619615 MemQueuePeek: queue empty [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.405 [npu_driver.cc:4330] 619619 MemQueuePeek: peek mem queue, deviceId=3, qid=2, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.412 [tensor_data_transfer.cpp:1054]619621 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.420 [api_impl.cc:6121] 619623 MemQueuePeek: Start to peek queue on devId 3, qid is 4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.429 [tensor_data_transfer.cpp:1054]619617 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.436 [tensor_data_transfer.cpp:612]619615 acltdtReceiveTensorV2: queue is empty, device is 3, name is _npu_log [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.448 [tensor_data_transfer.cpp:1056]619621 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.456 [npu_driver.cc:4330] 619623 MemQueuePeek: peek mem queue, deviceId=3, qid=4, timeout=800. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.465 [tensor_data_transfer.cpp:1056]619617 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.482 [api_impl.cc:6121] 619621 MemQueuePeek: Start to peek queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.497 [api_impl.cc:6121] 619617 MemQueuePeek: Start to peek queue on devId 3, qid is 1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.506 [npu_driver.cc:4330] 619621 MemQueuePeek: peek mem queue, deviceId=3, qid=3, timeout=800. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.513 [npu_driver.cc:4330] 619617 MemQueuePeek: peek mem queue, deviceId=3, qid=1, timeout=800. [INFO] DEVICE(619431,fffef847dfa0,python3.9):2025-04-03-09:35:52.900.613 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.900.729 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is _npu_log [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.752 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:52.900.768 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=0. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:52.901.196 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is _npu_log [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.901.231 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:52.901.262 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619431,python3.9):2025-04-03-09:35:53.732.170 [queue_client_comm.c:114][ascend][curpid:619431,619621][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:53.732.170 [queue_client_comm.c:114][ascend][curpid:619431,619619][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:53.732.170 [queue_client_comm.c:114][ascend][curpid:619431,619623][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:53.732.169 [queue_client_comm.c:114][ascend][curpid:619431,619617][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619431,python3.9):2025-04-03-09:35:53.732.202 [queue_client_comm.c:114][ascend][curpid:619431,619625][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.288 [npu_driver.cc:4338] 619621 MemQueuePeek: queue empty [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.299 [npu_driver.cc:4338] 619619 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.307 [tensor_data_transfer.cpp:612]619621 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_image_summary [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.312 [tensor_data_transfer.cpp:612]619619 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.337 [npu_driver.cc:4338] 619623 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.369 [tensor_data_transfer.cpp:612]619623 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_scalar_summary [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.378 [npu_driver.cc:4338] 619617 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.411 [tensor_data_transfer.cpp:612]619617 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_tensor_dump [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.424 [npu_driver.cc:4338] 619625 MemQueuePeek: queue empty [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.453 [tensor_data_transfer.cpp:612]619625 acltdtReceiveTensorV2: queue is empty, device is 3, name is ms_histogram_summary [INFO] DEVICE(619431,fffedafdefa0,python3.9):2025-04-03-09:35:53.732.528 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619431,fffef13e8fa0,python3.9):2025-04-03-09:35:53.732.563 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619431,fffed9fbefa0,python3.9):2025-04-03-09:35:53.732.548 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619431,fffedbffefa0,python3.9):2025-04-03-09:35:53.732.528 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619431,fffed8f9efa0,python3.9):2025-04-03-09:35:53.732.667 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.732.915 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is ms_histogram_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.947 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.732.961 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=5. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.733.400 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is ms_histogram_summary [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.733.432 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.733.456 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.733.497 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is ms_scalar_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.733.514 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 4. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.733.527 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=4. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.733.927 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is ms_scalar_summary [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.733.953 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.733.971 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.734.044 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is ms_image_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.734.061 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.734.073 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.734.444 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is ms_image_summary [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.734.469 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.734.488 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.734.538 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is ms_tensor_summary [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.734.555 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.734.568 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=2. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.734.923 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is ms_tensor_summary [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.734.947 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.734.972 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.735.020 [tensor_data_transfer.cpp:973]619431 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 3, name is ms_tensor_dump [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.735.036 [api_impl.cc:6065] 619431 MemQueueDestroy: Start to destroy queue on devId 3, qid is 1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.735.048 [npu_driver.cc:3989] 619431 MemQueueDestroy: deviceId=3, qid=1. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.735.404 [tensor_data_transfer.cpp:977]619431 acltdtDestroyChannel: acltdtDestroyChannel success, device is 3, name is ms_tensor_dump [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.735.424 [device.cpp:183]619431 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.735.442 [api_impl.cc:3419] 619431 DeviceReset: reset device, device_id=3. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.735.455 [runtime.cc:3444] 619431 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.735.467 [api_impl.cc:3449] 619431 DeviceReset: Succ, userDeviceId=3. [INFO] ASCENDCL(619431,python3.9):2025-04-03-09:35:53.735.479 [device.cpp:189]619431 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 3. [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.501 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.524 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.557 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.574 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.592 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_3 [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.607 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.645 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.661 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.735.683 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.735.928 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.735.956 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.735.973 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.735.989 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.736.012 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.736.042 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.736.064 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.736.079 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(619431,fffef0bd8fa0,python3.9):2025-04-03-09:35:53.736.118 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] PS(619431,ffff2afdefa0,python3.9):2025-04-03-09:35:53.736.102 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] PS(619431,ffff2b7eefa0,python3.9):2025-04-03-09:35:53.736.112 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] DISTRIBUTED(619431,fffebd7aefa0,python3.9):2025-04-03-09:35:53.736.186 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.276 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.298 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.314 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.333 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.347 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.363 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.378 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.509 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.545 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.582 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.600 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.615 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.632 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.736.647 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.034 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.052 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.077 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.091 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.120 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.136 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.383 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.400 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.429 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.446 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.461 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.476 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:53.737.599 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:53.922.324 [awatchdog_monitor.c:160](tid:619467) thread 619631 not exist [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.964.751 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:53.964.813 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:53.964.829 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:53.964.841 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.964.872 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:53.964.989 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:54.243.497 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:54.243.610 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.629.725 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.085 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.121 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.165 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.180 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.656 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.677 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.691 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.705 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.730 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.630.745 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.084 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.102 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.147 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.162 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.225 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.239 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.824 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.845 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.631.998 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.015 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.114 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.130 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.144 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.170 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.191 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.204 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.217 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.228 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.239 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.250 [heartbeat.cc:145] [619431][DeInit]heartbeat deinit begin [INFO] HCCL(619431,python3.9):2025-04-03-09:35:54.632.263 [heartbeat.cc:183] [619431][DeInit]heartbeat deinit end [INFO] DEVICE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:54.632.710 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 3, group name: hccl_world_group [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:54.632.798 [hash_data.cpp:63] >>> (tid:619431) HashData uninitialize success [INFO] PROFILING(619431,python3.9):2025-04-03-09:35:54.632.822 [msprof_reporter_mgr.cpp:270] >>> (tid:619431) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:54.635.905 [api_impl.cc:2003] 619431 DevFree: device free mem=0x12c041200000 [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:54.637.763 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:54.637.797 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:54.637.983 [adx_dump_record.cpp:229][tid:619431] start to dump uninit [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:54.638.011 [adx_dump_record.cpp:239][tid:619431] dump uninit success [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:54.638.029 [adx_dump_record.cpp:674][tid:619613] exit record file thread [INFO] GE(619431,python3.9):2025-04-03-09:35:54.900.950 [operator_factory_impl.cc:392]619431 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.193 [operator_factory_impl.cc:392]619431 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.245 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.260 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.276 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.287 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.299 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.309 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.320 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:54.944.333 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:55.093.792 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:55.094.004 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] GE(619431,python3.9):2025-04-03-09:35:55.094.054 [opsproto_manager.cc:66]619431 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.180.755 [runtime.cc:5239] 619582 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:55.180.808 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:55.180.824 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619431,python3.9):2025-04-03-09:35:55.180.839 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619582] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.180.870 [runtime.cc:3077] 619582 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=3, devId=3, tdt status=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.180.990 [npu_driver.cc:551] 619582 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.292.802 [api_impl.cc:7151] 619431 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.293.029 [api_impl.cc:7135] 619431 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.301.188 [api_impl.cc:7151] 619431 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.301.356 [api_impl.cc:7135] 619431 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.309.164 [api_impl.cc:7121] 619431 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.312.872 [api_impl.cc:7121] 619431 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:55.321.118 [osal.cc:321] 619431 ThreadExit: ThreadExit start. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:56.396.727 [stars_engine.cc:1833] 619582 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:56.396.789 [stars_engine.cc:1836] 619582 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619431,ffff9a7b3020,python3.9):2025-04-03-09:35:56.397.018 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619431,python3.9):2025-04-03-09:35:56.455.368 [ops_kernel_builder_registry.cc:71]619431 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619431,python3.9):2025-04-03-09:35:56.455.448 [ops_kernel_builder_registry.cc:41]619431 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:57.380.219 [adx_server_manager.cpp:236][tid:619614] server manager stop [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:57.805.804 [adx_hdc_epoll.cpp:57][tid:619431] start to close hdc epoll [INFO] GE(619431,python3.9):2025-04-03-09:35:57.881.194 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaaf947d4a0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.886.460 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaaf9481ad0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.889.572 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaaf94abf50]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.893.887 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaaf947eae0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.895.473 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaafa8b80d0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.897.019 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaafa8e00c0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.898.315 [op_impl_registry_holder_manager.cc:31]619431 CloseHandle:start close handle, handle[0xaaaafa9a3740]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.918.732 [mem_manager.cc:118]619431 Finalize:MemManager Finalize. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.918.769 [session_scope_mem_allocator.cc:73]619431 FreeAllMemory:Free all memory [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.260 [session_scope_mem_allocator.cc:73]619431 FreeAllMemory:Free all memory [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.291 [caching_allocator.cc:474]619431 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.320 [caching_allocator.cc:391]619431 FreeBlocks:Free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.333 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.343 [caching_allocator.cc:422]619431 FreeBlockBins:Free block bins. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.355 [caching_allocator.cc:474]619431 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.367 [caching_allocator.cc:391]619431 FreeBlocks:Free blocks. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.376 [caching_allocator.cc:358]619431 FreeCachedBlocks:Free cached blocks [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.387 [caching_allocator.cc:422]619431 FreeBlockBins:Free block bins. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.409 [mem_manager.cc:145]619431 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.425 [rts_caching_mem_allocator.cc:150]619431 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619431,python3.9):2025-04-03-09:35:57.921.473 [rts_caching_mem_allocator.cc:136]619431 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.415 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.476 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.515 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.527 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.539 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.548 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.560 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.569 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.580 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.590 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.598 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.606 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.615 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.624 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.632 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] HCCL(619431,python3.9):2025-04-03-09:35:58.013.641 [network_manager.cc:875] [619431]destroy all nic/vnic. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.520 [runtime.cc:1570] 619431 ~Runtime: deconstruct runtime [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.554 [runtime.cc:1577] 619431 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.864 [raw_device.cc:262] 619431 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.896 [engine.cc:188] 619431 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.920 [ringbuffer_maintain_task.cc:83] 619431 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.934 [stream.cc:1400] 619431 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.950 [stream.cc:2595] 619431 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=3 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.973 [npu_driver.cc:751] 619431 SqTaskSend: device_id=3, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.019.986 [stars_engine.cc:893] 619431 SendTask: Sq task send finished, device_id=3, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.001 [stream.cc:1494] 619431 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.018 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.032 [stream.cc:2886] 619431 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.045 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.056 [stream.cc:2886] 619431 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.067 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.082 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.106 [task.cc:185] 619431 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.119 [logger.cc:152] 619431 TaskFinished: device_id=3, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.137 [stars_engine.cc:437] 619431 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.150 [stream.cc:2886] 619431 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.177 [stars_engine.cc:524] 619431 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.188 [npu_driver.cc:5547] 619431 GetSqHead: deviceId=3, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.203 [stream.cc:852] 619431 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.215 [stream.cc:938] 619431 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.020.227 [stream.cc:1507] 619431 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.684 [stream_sqcq_manage.cc:451] 619431 FreeLogicCq: Return(0), threadIdentifier(281473273507872), devId(3), tsId(0), cqId(1047), isFastCq(0). [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.731 [npu_driver.cc:6163] 619431 StreamUnBindLogicCq: success: deviceId=3, tsId=0, streamId=0, logicCq=1047. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.783 [npu_driver.cc:1180] 619431 LogicCqFree: Succ:device_id=3,ts_id=0,type=2,logic_cq_id=1047, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.799 [stream.cc:237] 619431 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.813 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.023.824 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.452 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.471 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.488 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.506 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=0, device_id=3, tsId=0, streamFlag=192 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.593 [stars_engine.cc:206] 619583 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.726 [stars_engine.cc:191] 619431 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.741 [osal.cc:225] 619431 Join: mmJoinTask failed, thread=281471627206560 does not exist! [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.752 [stars_engine.cc:150] 619431 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.781 [runtime.cc:5239] 619431 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=3, deviceId=3. [INFO] TDT(619431,python3.9):2025-04-03-09:35:58.024.798 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619431] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.813 [runtime.cc:3003] 619431 StopAicpuExecutor: TsdClose success, userDeviceId=3, devId=3, tdt status=0 [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.024.839 [atrace_api.c:118](tid:619431) AtraceDestroy start. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.024.856 [tracer_mgr_operate.c:248](tid:619431) destroy object RUNTIME_ATRACE_DEV3_TS0, exitSave(false). [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.024.873 [atrace_api.c:120](tid:619431) AtraceDestroy end. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.887 [stream.cc:237] 619431 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.899 [stream_sqcq_manage.cc:216] 619431 DeAllocStreamSqCq: deviceId=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.024.910 [npu_driver.cc:966] 619431 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.025.114 [npu_driver.cc:974] 619431 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=3, tsId=0, drvFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.025.129 [stream_sqcq_manage.cc:228] 619431 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.025.141 [npu_driver.cc:432] 619431 StreamIdFree: stream id free begin, device_id=3, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.025.155 [npu_driver.cc:441] 619431 StreamIdFree: Stream id free success, stream_id=1, device_id=3, tsId=0, streamFlag=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.037.474 [stars_engine.cc:58] 619431 ~StarsEngine: Destructor. [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.037.503 [engine.cc:84] 619431 ~Engine: Destructor. [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.335 [trs_dev_drv.c:370][ascend][curpid:619431,619431][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=3; fd=37; mmapFd=38) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.359 [devmm_svm.c:2238][ascend][curpid:619431,619431][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.375 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.393 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.408 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.423 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.436 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev3 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.452 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev3 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.468 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev3 Cached_size:1961984Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.480 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev3 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.506 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev3 Cached_size:2097152Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.519 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev3 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.535 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev3 Cached_size:2097152Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.549 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev3 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.563 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev3 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.578 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev3 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.593 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev3 Cached_size:23068672Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.605 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev3 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.619 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev3 Cached_size:6291456Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.631 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev3 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.645 [svm_mem_statistics.c:140][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev3 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.038.660 [svm_mem_statistics.c:146][ascend][curpid:619431,619431][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev3 Cached_size:16777216Bytes [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.112.685 [drv_log_user.c:623][ascend][curpid:619431,619431][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=3; phyid=3; vfid=0; hostpid=619431; devpid=31148) [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.112.721 [npu_driver.cc:3045] 619431 DeviceClose: Close device success, device_id=3, tsId=0 [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.112.737 [raw_device.cc:125] 619431 ~Device: deconstruct device [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.223.122 [plugin_old_arch.cc:25] 619431 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.223.152 [runtime_keeper.cc:167] 619431 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619431,python3.9):2025-04-03-09:35:58.223.963 [task_fail_callback_data_manager.cc:63] 619431 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.305.403 [stacktrace_signal.c:188](tid:619431) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.382.169 [atrace_client_communication.c:141](tid:619431) send [end] msg. [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.383.916 [hdc_core.c:1982][ascend][curpid:619431,619431][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4288; pid=619431) [INFO] IDEDD(619431,python3.9):2025-04-03-09:35:58.383.947 [hdc_api.cpp:671][tid:619431] connect succ, peer_node: 0, peer_devid: 3 [INFO] DRV(619431,python3.9):2025-04-03-09:35:58.384.312 [hdc_client.c:417][ascend][curpid:619431,619431][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4288) [INFO] ATRACE(619431,python3.9):2025-04-03-09:35:58.384.382 [atrace_client_communication.c:164](tid:619431) send [end] msg success. output_allreduce/worker_5.log0000644000175100017500000700124414773362776017130 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619439,ffff81bdb020,python3.9):2025-04-03-09:35:26.940.351 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:26.954.086 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 5 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.419.908 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.419.986 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.033 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.053 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.072 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.091 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.111 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.129 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.148 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.202 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.215 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.229 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.240 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.259 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.275 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.295 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.311 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.326 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.344 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.357 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.371 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.397 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.412 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.434 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.451 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.470 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.497 [ir_data_type_symbol_store.cc:240]619439 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.517 [ir_data_type_symbol_store.cc:240]619439 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.535 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.551 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.566 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.586 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.601 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.615 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.631 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.647 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.662 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.678 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.694 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.707 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.726 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.740 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.755 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.777 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.793 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.808 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.824 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.836 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.853 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.868 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.883 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.901 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.917 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.931 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.948 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.964 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.980 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.420.997 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.013 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.030 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.046 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.059 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.075 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.090 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.106 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.123 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.138 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.151 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.188 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.204 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.219 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.236 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.252 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.267 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.289 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.305 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.320 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.336 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.352 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.364 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.381 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.398 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.411 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.429 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.445 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.456 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.471 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.487 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.504 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.516 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.530 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.547 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.561 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.576 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.598 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.615 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.632 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.647 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.660 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.675 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.691 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.702 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.716 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.733 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.751 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.766 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.779 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.794 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.809 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.820 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.835 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.851 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.867 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.882 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.897 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.912 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.926 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.940 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.954 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.978 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.421.992 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.010 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.025 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.037 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.053 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.069 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.086 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.097 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.109 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.123 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.141 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.156 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.168 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.183 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.198 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.213 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.229 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.240 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.255 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.270 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.285 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.300 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.312 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.325 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.341 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.364 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.380 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.392 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.407 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.423 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.436 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.452 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.464 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.478 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.493 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.509 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.525 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.543 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.564 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.580 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.597 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.615 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.630 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.642 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.656 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.672 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.686 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.701 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.716 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.737 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.749 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.763 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.780 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.797 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.812 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.826 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.842 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.858 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.873 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.889 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.907 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.922 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.939 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.968 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.984 [ir_data_type_symbol_store.cc:173]619439 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619439,python3.9):2025-04-03-09:35:27.422.996 [ir_data_type_symbol_store.cc:219]619439 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619439,python3.9):2025-04-03-09:35:27.423.015 [ir_data_type_symbol_store.cc:182]619439 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.429.976 [awatchdog_core.c:99](tid:619439) watchdog get platform host side [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.119 [awatchdog_monitor.c:127](tid:619439) create monitor thread successfully, tid :281472221208480 [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.142 [awatchdog_monitor.c:226](tid:619439) AwdMonitorInit [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.333 [trace_attr.c:99](tid:619439) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.370 [trace_attr.c:189](tid:619439) attr init success, timeout=0ms. [WARNING] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.387 [trace_system_api.c:115](tid:619439) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.445 [trace_recorder.c:40](tid:619439) home_directory=/home/jenkins. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.462 [trace_recorder.c:121](tid:619439) use root path: /home/jenkins/ascend [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.477 [trace_event.c:151](tid:619439) init trace event success [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.430.524 [atrace_client_core.c:185](tid:619439) atrace client init success. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.831 [stacktrace_dumper.c:218](tid:619439) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.854 [trace_system_api.c:115](tid:619439) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.929 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 2 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.947 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 15 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.964 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 3 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.978 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 4 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.431.991 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 5 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.004 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 6 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.017 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 7 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.032 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 8 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.045 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 11 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.061 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 24 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.074 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 25 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.086 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 31 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.101 [stacktrace_signal.c:165](tid:619439) register signal handler for signo 35 succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.113 [stacktrace_signal.c:168](tid:619439) trace signal init succeed. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:27.432.126 [trace_core.c:51](tid:619439) trace init success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.432.900 [task_fail_callback_data_manager.cc:58] 619439 TaskFailCallBackManager: Constructor. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.442.948 [operator_factory_impl.cc:232]619439 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619439,python3.9):2025-04-03-09:35:27.442.988 [operator_factory_impl.cc:246]619439 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619439,python3.9):2025-04-03-09:35:27.443.004 [operator_factory_impl.cc:239]619439 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619439,python3.9):2025-04-03-09:35:27.450.409 [adapter_rts.cc:2720] [619439][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.465.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.465.397 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:27.465.427 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:27.475.496 [prof_cann_plugin.cpp:256] (tid:619439) Register module[6] callback. [INFO] OP(619439,python3.9):2025-04-03-09:35:27.526.087 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619439] cachelimit is 10000 [INFO] OP(619439,python3.9):2025-04-03-09:35:27.530.220 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619439] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619439,python3.9):2025-04-03-09:35:27.530.274 [op_dfx.cpp:54][NNOP][SystemConfig][619439] enable debug kernel flag 0. [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:27.530.296 [prof_cann_plugin.cpp:256] (tid:619439) Register module[61] callback. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.584.500 [mem_manager.cc:138]619439 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.591.676 [cache_policy.cc:47]619439 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619439,python3.9):2025-04-03-09:35:27.606.784 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.606.919 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.024 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.052 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.075 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.096 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.120 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.141 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.163 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.186 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.207 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.229 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.250 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.272 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.293 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.314 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.334 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.358 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.378 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.399 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.503 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.614 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.638 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.739 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.765 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.787 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.808 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.829 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.853 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.875 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.897 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.919 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.941 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.962 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.607.983 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.004 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.025 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.047 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.068 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.091 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.113 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.134 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.283 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.394 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.494 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.594 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.697 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.726 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.830 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.933 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.608.958 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.052 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.171 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.194 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.216 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.238 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.259 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.282 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.304 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.326 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.347 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.370 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.391 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.411 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.433 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.457 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.559 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.666 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.764 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.860 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.887 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.908 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.609.938 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.037 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.137 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.162 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.271 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.298 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.395 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.494 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.593 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.610.691 [addn.cc:29]619439 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.053 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.159 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.258 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.356 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.451 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.545 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.640 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.733 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.750 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.838 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619439,python3.9):2025-04-03-09:35:27.611.937 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619439,python3.9):2025-04-03-09:35:27.612.032 [symbolic_kernel_factory.cc:27]619439 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619439,python3.9):2025-04-03-09:35:27.618.899 [process_node_engine_manager.cc:140]619439 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619439,python3.9):2025-04-03-09:35:27.619.033 [process_node_engine_manager.cc:140]619439 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619439,python3.9):2025-04-03-09:35:27.625.692 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619439,python3.9):2025-04-03-09:35:27.625.729 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619439,python3.9):2025-04-03-09:35:27.626.258 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619439] start packet count timer [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:27.648.354 [prof_atls_plugin.cpp:117] (tid:619439) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:27.648.394 [prof_atls_plugin.cpp:117] (tid:619439) RegisterProfileCallback, callback type is 8 [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.725.029 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.725.535 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.774.262 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.774.418 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 5 [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.774.462 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.775.278 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.776.965 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.048 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.089 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.181 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.215 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.275 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.777.302 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.779.011 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.779.043 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.779.083 [acl.cpp:753]619439 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.779.164 [runtime_keeper.cc:78] 619439 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.782.999 [plugin_old_arch.cc:16] 619439 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaab0b946bc0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.034 [runtime_keeper.cc:149] 619439 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.044 [runtime_keeper.cc:221] 619439 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.084 [runtime.cc:1387] 619439 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.114 [runtime.cc:1163] 619439 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.139 [npu_driver.cc:4062] 619439 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.160 [runtime.cc:5096] 619439 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.176 [runtime.cc:1173] 619439 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.193 [runtime.cc:1183] 619439 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.212 [runtime.cc:1193] 619439 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.227 [profiler.cc:84] 619439 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.248 [runtime.cc:1209] 619439 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.273 [runtime.cc:1219] 619439 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.285 [runtime.cc:1230] 619439 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.298 [runtime.cc:1242] 619439 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.313 [runtime.cc:1264] 619439 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.327 [runtime.cc:1270] 619439 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.437 [runtime.cc:109] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.839 [runtime.cc:121] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.921 [runtime.cc:1314] 619439 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.783.938 [runtime.cc:1322] 619439 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.065 [runtime.cc:1338] 619439 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.128 [utils.cc:105] 619439 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.222 [runtime.cc:1664] 619439 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.264 [runtime.cc:1688] 619439 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.287 [runtime.cc:1953] 619439 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.315 [runtime.cc:1477] 619439 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:27.784.337 [prof_atls_plugin.cpp:210] (tid:619439) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.359 [heterogenous.cc:19] 619439 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:27.784.391 [api_c_soc.cc:70] 619439 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.784.408 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.784.422 [acl.cpp:760]619439 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.784.461 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.821.837 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.857.077 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.857.547 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.857.993 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.858.834 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.859.285 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.861.139 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.861.185 [acl.cpp:753]619439 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.861.218 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:27.861.233 [acl.cpp:760]619439 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:27.861.260 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.817.309 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.819.185 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.842.221 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.851.514 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.855.066 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.858.092 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.886.746 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619439,python3.9):2025-04-03-09:35:28.902.684 [platform_info.cpp:853]619439 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.835.851 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.837.761 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.837.902 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.840.236 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.840.380 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.840.498 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.840.638 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.840.747 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.841.897 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.842.036 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.843.980 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.844.114 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.844.247 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.844.363 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.844.468 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619439,python3.9):2025-04-03-09:35:29.844.576 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619439,python3.9):2025-04-03-09:35:30.501.808 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619439,python3.9):2025-04-03-09:35:34.880.956 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.147.448 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.147.561 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.147.606 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619439,python3.9):2025-04-03-09:35:35.198.454 [ops_kernel_builder_registry.cc:55]619439 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619439,python3.9):2025-04-03-09:35:35.198.614 [ops_kernel_builder_registry.cc:28]619439 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.239.474 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.239.619 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.239.723 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.239.747 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:35.239.765 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619439,python3.9):2025-04-03-09:35:36.078.722 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:37.848.946 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:37.849.611 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:37.861.056 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.952.434 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:37.952.505 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.953.790 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.953.835 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.953.857 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.953.887 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.132 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471246491552 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.250 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471238037408 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.275 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.408 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471229583264 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.620 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471017283488 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.653 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.674 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.923 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58728, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:37.954.968 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [WARNING] DISTRIBUTED(619439,ffff20a75fa0,python3.9):2025-04-03-09:35:37.954.989 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58728 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.060 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.088 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.107 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.124 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.234 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58764, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.455.255 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619439,ffff21a95fa0,python3.9):2025-04-03-09:35:38.455.270 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58764 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.955.334 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.955.357 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.955.609 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 5 has been registered successfully. [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:38.955.737 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619439,ffff137eefa0,python3.9):2025-04-03-09:35:38.955.791 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619439,ffff137eefa0,python3.9):2025-04-03-09:35:38.955.817 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:39.455.817 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:39.955.907 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.455.987 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(4/1200). [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.090 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3002.24 msec. [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.119 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.140 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.168 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 5 rank id: 5 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.190 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.233 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 9398 to 9653 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.257 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.114 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.286 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.310 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3002.57 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.333 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.472 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.492 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.516 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.543 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.561 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.956.578 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.958.246 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.958.269 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 9398 to 9653 [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.958.454 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:82c36f05-3977-4032-a338-e867f190a801, the ip:127.0.0.1, the port:9398 [INFO] PS(619439,ffff12fdefa0,python3.9):2025-04-03-09:35:40.958.480 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619439,ffff12fdefa0,python3.9):2025-04-03-09:35:40.958.513 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.090 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.248 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_5 has been registered successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.274 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.822 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 5 has been started successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.848 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.892 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.913 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.947 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 5, global rank size: 8 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.974 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 4.615 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.960.995 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 5 is devserver-c233-2 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.125 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.199 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.224 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 5 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.242 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 5 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.269 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [5]. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.292 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.295 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.370 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.419 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.447 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:40.961.484 [device.cpp:365]619439 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:40.961.645 [api_impl.cc:249] 619439 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.961.668 [api_impl.cc:6795] 619439 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:40.961.684 [device.cpp:371]619439 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:40.961.761 [callback.cpp:37]619439 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:40.961.801 [callback.cpp:44]619439 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.827 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:40.961.856 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:40.961.898 [device.cpp:143]619439 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.962.008 [npu_driver.cc:124] 619439 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.962.043 [api_impl.cc:2999] 619439 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.962.085 [runtime.cc:3602] 619439 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.962.353 [hdc_core.c:1982][ascend][curpid:619439,619439][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4214; pid=619439) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.962.483 [runtime.cc:3609] 619439 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.962.532 [atrace_client_communication.c:103](tid:619439) send [hello] msg. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.962.551 [atrace_client_communication.c:63](tid:619439) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.962.681 [hdc_core.c:1982][ascend][curpid:619439,619439][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4215; pid=619439) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:40.962.702 [hdc_api.cpp:671][tid:619439] connect succ, peer_node: 0, peer_devid: 5 [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.964.060 [hdc_client.c:417][ascend][curpid:619439,619439][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4215) [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.097 [atrace_client_communication.c:65](tid:619439) send msg by adx[finish], ret=0. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.113 [atrace_client_communication.c:97](tid:619439) msgType=1, magic=49959, version=1. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.130 [atrace_client_communication.c:135](tid:619439) send [hello] msg success. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.232 [atrace_client_core.c:160](tid:619439) start to receive device[5] trace log. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.964.275 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.964.298 [runtime.cc:2855] 619439 startAicpuExecutor: TsdOpen devId=5, device mode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:40.964.315 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.964.368 [hdc_core.c:1982][ascend][curpid:619439,619545][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4216; pid=619439) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.368 [client_manager.cpp:160][GetPlatformInfo][tid:619439] [TsdClient] mode[1] chipType[5] [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:40.964.402 [hdc_api.cpp:671][tid:619545] connect succ, peer_node: 0, peer_devid: 5 [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.964.432 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619439] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.450 [client_manager.cpp:582][GetVisibleDevices][tid:619439] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.466 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.481 [client_manager.cpp:125][GetInstance][tid:619439] [ClientManager] GetInstance, deviceId[5], g_runningMode[0], begin saving instance [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.522 [client_manager.cpp:435][IsHostEnvironment][tid:619439] [TsdClient] logicDeviceId is [5], hostaicpunum[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.540 [client_manager.cpp:464][GetClientRunMode][tid:619439] runningMode:0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.555 [client_manager.cpp:127][GetInstance][tid:619439] [ClientManager] Current mode:1 [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.964.569 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619439] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.583 [client_manager.cpp:347][GetProfilingMode][tid:619439] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.964.624 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619439] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.640 [process_mode_base.cpp:149][GetLogLevel][tid:619439] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.652 [atrace_client_communication.c:177](tid:619545) send [start] msg. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:40.964.700 [atrace_client_communication.c:189](tid:619545) send [start] msg success. [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.733 [process_mode_base.cpp:161][GetLogLevel][tid:619439] [TsdClient] Get Log Level[001] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.752 [process_mode_manager.cpp:71][OpenProcess][tid:619439] [ProcessModeManager] enter into open process deviceId[5] rankSize[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.769 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619439] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.841 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[.], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.916 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[scene.info], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.964.982 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.064 [client_manager.cpp:264][CheckPackageExists][tid:619439] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.091 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[ascend_install.info], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.151 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[version.info], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.213 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[..], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.275 [client_manager.cpp:261][CheckPackageExists][tid:619439] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.406 [client_manager.cpp:268][CheckPackageExists][tid:619439] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.423 [client_manager.cpp:283][CheckPackageExists][tid:619439] [TsdClient][deviceId=5] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.440 [client_manager.cpp:286][CheckPackageExists][tid:619439] [TsdClient][deviceId=5] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.965.506 [devdrv_hccl.c:103][ascend][curpid:619439,619439][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619439) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.524 [process_mode_manager.cpp:396][InitTsdClient][tid:619439] [TsdClient][deviceId=5] get process sign success, procPid[619439] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.540 [process_mode_manager.cpp:403][InitTsdClient][tid:619439] [TsdClient] deviceId[5] begin to init hdc client [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.575 [hdc_common.cpp:67][InitMsgSize][tid:619439] HdcCommon::InitMsgSize Start [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.591 [hdc_common.cpp:80][InitMsgSize][tid:619439] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.610 [hdc_client.cpp:184][CreateHdcSession][tid:619439] HdcClient::CreateHdcSession Start [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.697 [hdc_client.cpp:208][CreateHdcSession][tid:619439] [HdcClient] deviceId: 5 connect session [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.965.732 [hdc_core.c:1982][ascend][curpid:619439,619439][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4217; pid=619439) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.749 [hdc_client.cpp:219][CreateHdcSession][tid:619439] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.763 [hdc_client.cpp:230][CreateHdcSession][tid:619439] [HdcClient] deviceId: 5 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.783 [version_verify.cpp:36][SetVersionInfo][tid:619439] VersionVerify: send client version to server [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.833 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.859 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.965.875 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.965.921 [internal_api.cpp:101][GetScheduleEnv][tid:619439] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.281 [version_verify.cpp:68][PeerVersionCheck][tid:619439] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.303 [version_verify.cpp:89][ParseVersionInfo][tid:619439] VersionVerify: pass client version info success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.318 [hdc_client.cpp:275][CheckHdcConnection][tid:619439] Service[2] create hdc success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.336 [version_verify.cpp:122][SpecialFeatureCheck][tid:619439] VersionVerify: new type[35], supported [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.966.354 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619439] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.389 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619439] [TsdClient][deviceId=5] [sessionId=1] wait package info respond [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.966.406 [hdc_client.cpp:154][TsdRecvData][tid:619439] recv timeout is:150000 ms [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.369 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619439] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.386 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619439] [PackageInfoMsgProc] [sessionID=1] deviceId[5], checkCode[56684994] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.402 [hdc_client.cpp:172][TsdRecvData][tid:619439] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.418 [version_verify.cpp:122][SpecialFeatureCheck][tid:619439] VersionVerify: new type[64], supported [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.436 [hdc_client.cpp:411][Destroy][tid:619439] enter HdcClient::Destroy() function [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.967.451 [hdc_client.c:417][ascend][curpid:619439,619439][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4217) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.480 [hdc_client.cpp:394][DestroyClient][tid:619439] begin drvHdcClientDestroy [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.498 [hdc_client.cpp:399][DestroyClient][tid:619439] end drvHdcClientDestroy [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.511 [hdc_client.cpp:377][ClearClientPtr][tid:619439] begin HdcClient::ClearClientPtr [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.525 [hdc_client.cpp:385][ClearClientPtr][tid:619439] end HdcClient::ClearClientPtr [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.538 [hdc_client.cpp:418][Destroy][tid:619439] end HdcClient::Destroy() function [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.567 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619439] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.586 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619439] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.967.606 [devdrv_hccl.c:103][ascend][curpid:619439,619439][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619439) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.622 [process_mode_manager.cpp:396][InitTsdClient][tid:619439] [TsdClient][deviceId=5] get process sign success, procPid[619439] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.636 [process_mode_manager.cpp:403][InitTsdClient][tid:619439] [TsdClient] deviceId[5] begin to init hdc client [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.653 [hdc_common.cpp:67][InitMsgSize][tid:619439] HdcCommon::InitMsgSize Start [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.668 [hdc_common.cpp:80][InitMsgSize][tid:619439] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.692 [hdc_client.cpp:184][CreateHdcSession][tid:619439] HdcClient::CreateHdcSession Start [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.749 [hdc_client.cpp:208][CreateHdcSession][tid:619439] [HdcClient] deviceId: 5 connect session [INFO] DRV(619439,python3.9):2025-04-03-09:35:40.967.780 [hdc_core.c:1982][ascend][curpid:619439,619439][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4218; pid=619439) [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.797 [hdc_client.cpp:219][CreateHdcSession][tid:619439] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.809 [hdc_client.cpp:230][CreateHdcSession][tid:619439] [HdcClient] deviceId: 5 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.826 [version_verify.cpp:36][SetVersionInfo][tid:619439] VersionVerify: send client version to server [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.839 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.855 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.967.869 [version_verify.cpp:52][SetVersionInfo][tid:619439] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.117 [version_verify.cpp:68][PeerVersionCheck][tid:619439] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.137 [version_verify.cpp:89][ParseVersionInfo][tid:619439] VersionVerify: pass client version info success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.153 [hdc_client.cpp:275][CheckHdcConnection][tid:619439] Service[2] create hdc success [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.190 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619439] [TsdClient] tsd get process sign successfully, procpid[619439] signSize[48] [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.209 [version_verify.cpp:114][SpecialFeatureCheck][tid:619439] VersionVerify: previous type[6], supported [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.239 [process_mode_manager.cpp:88][OpenProcess][tid:619439] [ProcessModeManager] deviceId[5] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619439,python3.9):2025-04-03-09:35:40.968.257 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619439] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:40.968.268 [hdc_client.cpp:154][TsdRecvData][tid:619439] recv timeout is:150000 ms [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.167.609 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619439] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.167.635 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619439] [TsdClient] DeviceMsgProc recvMsg realDeviceId[5] msgType[8] localDevId[5] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.167.654 [hdc_client.cpp:172][TsdRecvData][tid:619439] [deviceId=5] the Tsd recv data already [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.167.680 [devmm_svm.c:132][ascend][curpid:619439,619439][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=5) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.169.863 [devmm_virt_interface.c:450][ascend][curpid:619439,619439][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.170.173 [drv_log_user.c:623][ascend][curpid:619439,619439][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=5; devid=5; vfid=0; hostpid=619439; devpid=31089) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.170.200 [devmm_svm.c:122][ascend][curpid:619439,619439][drv][devmm][devmm_set_device_info]Device info. (devid=5; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.170.484 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619439] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.170.505 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619439] enter into OpenInHost deviceid[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.170.532 [client_manager.cpp:435][IsHostEnvironment][tid:619439] [TsdClient] logicDeviceId is [5], hostaicpunum[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.170.549 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619439] host cpu not support [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.170.560 [process_mode_manager.cpp:121][OpenProcess][tid:619439] [TsdClient][deviceId=5] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.170.578 [runtime.cc:2879] 619439 startAicpuExecutor: userDeviceId=5, devId=5, tsdOpen ret=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.170.594 [runtime.cc:2891] 619439 startAicpuExecutor: TsdOpen success, devId=5, ret=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.170.609 [runtime.cc:3627] 619439 DeviceRetain: new dev ok id=5, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.170.626 [raw_device.cc:936] 619439 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.170.645 [npu_driver.cc:2968] 619439 DeviceOpen: Open device start, device_id=5, tsId=0. [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.170.736 [trs_sqcq.c:458][ascend][curpid:619439,619439][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=5) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.171.234 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=6; num=2044) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.171.694 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=7; num=2044) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.171.937 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=10; num=1024) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.185 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=11; num=1024) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.629 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=12; num=2048) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.655 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=8; num=16) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.684 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=9; num=16) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.703 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=13; num=4) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.172.724 [trs_sqcq.c:394][ascend][curpid:619439,619439][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=5; tsId=0; id_type=14; num=10) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.174.726 [trs_dev_drv.c:302][ascend][curpid:619439,619439][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=5; fd=36; mmapFd=37) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.174.748 [devmm_svm.c:132][ascend][curpid:619439,619439][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=5) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.174.782 [devmm_svm.c:122][ascend][curpid:619439,619439][drv][devmm][devmm_set_device_info]Device info. (devid=5; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.174.848 [npu_driver.cc:3011] 619439 DeviceOpen: Open device success, device_id=5, tsId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.174.867 [raw_device.cc:292] 619439 Init: chipType:0x5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.174.907 [raw_device.cc:304] 619439 Init: isAddrFlat:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.174.927 [raw_device.cc:320] 619439 Init: GetDevInfo, device_id=5, hardwareVersion=0x503 [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.174.945 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.174.962 [npu_driver.cc:1578] 619439 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.175.923 [raw_device.cc:352] 619439 Init: Get phy info, phy chip id 5, phy die id 0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.175.945 [runtime.cc:2771] 619439 GetTsdQos: [TSD] start get qos, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.175.960 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.175.974 [tsd_client.cpp:189][TsdCapabilityGet][tid:619439] TsdCapabilityGet Begin. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.175.989 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.004 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.017 [process_mode_manager.cpp:1042][CapabilityGet][tid:619439] [ProcessModeManager] enter into CapabilityGet process deviceId[5] type[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.033 [process_mode_manager.cpp:386][InitTsdClient][tid:619439] [TsdClient] tsd client has been inited before [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.048 [version_verify.cpp:114][SpecialFeatureCheck][tid:619439] VersionVerify: previous type[38], supported [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.084 [process_mode_manager.cpp:1065][CapabilityGet][tid:619439] [TsdClient][deviceId=5] [sessionId=1] [type=0]send capability success [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.103 [hdc_client.cpp:154][TsdRecvData][tid:619439] recv timeout is:150000 ms [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.577 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619439] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.596 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619439] [TsdClient] PidQosMsgProc recvMsg realDeviceId[5] msgType[39] localDevId[5] rspCode[0],pidqos[1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.614 [hdc_client.cpp:172][TsdRecvData][tid:619439] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.176.629 [process_mode_manager.cpp:1070][CapabilityGet][tid:619439] [TsdClient][logicDeviceId_=5][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.646 [runtime.cc:2782] 619439 GetTsdQos: [TSD] get qos success, userDeviceId=5, devId=5, qos=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.663 [engine.cc:78] 619439 Engine: Constructor. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.678 [stars_engine.cc:51] 619439 StarsEngine: Constructor. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.706 [engine_factory.cc:26] 619439 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.724 [engine_factory.cc:38] 619439 CreateEngine: Engine init success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.736 [raw_device.cc:364] 619439 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.750 [spm_pool.cc:42] 619439 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.865 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.883 [spm_pool.cc:55] 619439 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.897 [raw_device.cc:380] 619439 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.176.915 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.931 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.176.944 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.176.960 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.184.661 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.184.691 [task_allocator.cc:46] 619439 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.184.725 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.184.741 [h2d_copy_mgr.cc:45] 619439 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.184.758 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.184.774 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.185.847 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.185.870 [task_allocator.cc:46] 619439 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.185.884 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.185.900 [h2d_copy_mgr.cc:45] 619439 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.185.916 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.185.933 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.758 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.780 [task_allocator.cc:46] 619439 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.792 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.805 [h2d_copy_mgr.cc:45] 619439 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.832 [uma_arg_loader.cc:86] 619439 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.851 [task_allocator.cc:46] 619439 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.874 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.186.889 [uma_arg_loader.cc:93] 619439 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.687 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.706 [uma_arg_loader.cc:102] 619439 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.718 [uma_arg_loader.cc:104] 619439 Init: ALLOC PCIE is support[1] [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.187.733 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.750 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.876 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.893 [h2d_copy_mgr.cc:45] 619439 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.906 [raw_device.cc:393] 619439 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.920 [raw_device.cc:398] 619439 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.187.934 [task_allocator.cc:46] 619439 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.097 [buffer_allocator.cc:61] 619439 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.312 [raw_device.cc:109] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.684 [raw_device.cc:121] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.704 [raw_device.cc:425] 619439 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.762 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.840 [stream.cc:2246] 619439 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.876 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.188.901 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.495 [npu_driver.cc:6125] 619439 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.545 [npu_driver.cc:6137] 619439 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=366, streamId=0, logicCq=366, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.573 [npu_driver.cc:6189] 619439 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=0, logicCq=366. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.593 [npu_driver.cc:5916] 619439 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cd3bd000, addr=dfffcd3bd000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.619 [stream.cc:653] 619439 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.682 [stream.cc:276] 619439 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.701 [stream.cc:688] 619439 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=5, streamResId=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.721 [stream_state_callback_manager.cc:45] 619439 Notify: not reg stream state callback func! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.737 [raw_device.cc:593] 619439 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.757 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.777 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=16 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.792 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.881 [tsch_stream.cc:45] 619439 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=5 streamResId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.902 [raw_device.cc:1219] 619439 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.189.962 [osal.cc:309] 619439 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.065 [osal.cc:182] 619439 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.087 [osal.cc:309] 619439 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.155 [osal.cc:182] 619439 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.154 [stars_engine.cc:1785] 619546 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.174 [stars_engine.cc:201] 619547 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.201 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.190.221 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.190.240 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.190.251 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.272 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.326 [npu_driver.cc:6306] 619546 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.351 [stars_engine.cc:1855] 619546 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.367 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.485 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.509 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.548 [task_info.cc:1596] 619439 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.576 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.594 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.628 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.644 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.664 [stars_engine.cc:1122] 619439 SyncTask: Task Wait: device_id=5, stream_id=0, task_id=0, logicCqId=366. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.719 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.736 [stars_engine.cc:1071] 619439 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.768 [stars_engine.cc:1017] 619439 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.787 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.806 [stars_engine.cc:1027] 619439 ProcReport: taskType=97. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.830 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.845 [stars_engine.cc:1570] 619439 ProcLogicCqReport: CQE warning, device_id=5, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.866 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.883 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.900 [task_info.cc:1614] 619439 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.913 [raw_device.cc:1305] 619439 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.947 [stars_engine.cc:1351] 619439 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.969 [npu_driver.cc:5618] 619439 GetSqEnable: deviceId=5, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.984 [stars_engine.cc:1365] 619439 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.190.997 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.011 [npu_driver.cc:5595] 619439 GetSqTail: deviceId=5, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.027 [npu_driver.cc:5406] 619439 SetSqHead: deviceId=5, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.047 [npu_driver.cc:5357] 619439 EnableSq: deviceId=5, tsId=0, sqId=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.066 [stars_engine.cc:1430] 619439 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.085 [npu_driver.cc:5618] 619439 GetSqEnable: deviceId=5, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.108 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.121 [raw_device.cc:1331] 619439 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.191.136 [raw_device.cc:610] 619439 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.309 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.358 [ringbuffer_maintain_task.cc:98] 619439 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x1a40 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.378 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.391 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.411 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.426 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.444 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.458 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.490 [stars_engine.cc:622] 619439 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=366, retCode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.508 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.523 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.538 [stream.cc:2886] 619439 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.551 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.564 [stream.cc:2886] 619439 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.578 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.591 [stream.cc:2886] 619439 StarsWaitForTask: No.3 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.607 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.619 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.633 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.648 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.667 [stars_engine.cc:437] 619439 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.684 [stream.cc:2886] 619439 StarsWaitForTask: No.4 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.734 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.763 [task_info.cc:111] 619439 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.780 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.795 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.813 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.827 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.844 [stars_engine.cc:1122] 619439 SyncTask: Task Wait: device_id=5, stream_id=0, task_id=2, logicCqId=366. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.975 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.192.991 [stars_engine.cc:1071] 619439 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.003 [stars_engine.cc:1017] 619439 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.021 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.032 [stars_engine.cc:1027] 619439 ProcReport: taskType=29. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.045 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.059 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.073 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.096 [npu_driver.cc:5618] 619439 GetSqEnable: deviceId=5, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.109 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.193.122 [raw_device.cc:1232] 619439 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.193.145 [atrace_api.c:40](tid:619439) AtraceCreate start. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.268 [trace_rb_log.c:45](tid:619439) init ring buffer time finished, realTime 1743644141193206755, monotonicTime 4211040315280, freq 99993 kHz, minutesWest 480 minutes [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.287 [trace_rb_log.c:137](tid:619439) [RUNTIME_ATRACE_DEV5_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.308 [tracer_mgr_operate.c:66](tid:619439) create object RUNTIME_ATRACE_DEV5_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.326 [atrace_api.c:44](tid:619439) AtraceCreate end. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.375 [trace_event.c:70](tid:619439) limitedNum 1, processType 0 [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.391 [trace_event.c:372](tid:619439) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.409 [runtime.cc:3159] 619439 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619439,python3.9):2025-04-03-09:35:41.194.432 [client_manager.cpp:196][SetProfilingCallback][tid:619439] [TsdClient] set profiling callback success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.467 [runtime.cc:3364] 619439 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.492 [context.cc:109] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.545 [context.cc:121] 619439 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.561 [context.cc:276] 619439 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.606 [raw_device.cc:803] 619439 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.623 [npu_driver_dcache_lock.cpp:39] 619439 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.730 [raw_device.cc:528] 619439 AllocStackPhyBase: AllocStackPhyBase device_id=5, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.779 [raw_device.cc:232] 619439 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.803 [runtime.cc:3393] 619439 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.820 [runtime.cc:3560] 619439 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.836 [api_impl.cc:2987] 619439 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.853 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.867 [api_impl.cc:2992] 619439 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:41.194.892 [awatchdog_core.c:175](tid:619439) create watchdog for id : 65543, tid : 619439, timeout : 300s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.194.909 [api_impl.cc:3025] 619439 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.194.943 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.194.962 [acl.cpp:121]619439 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.473 [api_impl.cc:3476] 619439 GetDeviceInfo: get device info, deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.506 [api_impl.cc:3476] 619439 GetDeviceInfo: get device info, deviceId=5 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.197.560 [device.cpp:133]619439 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 5, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.197.581 [device.cpp:154]619439 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.197.603 [context.cpp:87]619439 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.197.629 [context.cpp:98]619439 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.197.657 [stream.cpp:48]619439 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.692 [api_impl.cc:1120] 619439 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.750 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.793 [stream.cc:2246] 619439 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.840 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.197.858 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.118 [npu_driver.cc:6125] 619439 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.164 [npu_driver.cc:6137] 619439 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=367, streamId=2, logicCq=367, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.186 [npu_driver.cc:6189] 619439 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=2, logicCq=367. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.205 [npu_driver.cc:5916] 619439 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c40cb000, addr=dfffc40cb000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.221 [stream.cc:653] 619439 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.198.271 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.289 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.198.447 [stream.cc:291] 619439 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.234 [stream.cc:688] 619439 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=5, streamResId=65535 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.255 [stream_state_callback_manager.cc:45] 619439 Notify: not reg stream state callback func! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.269 [api_impl.cc:1162] 619439 StreamCreate: Succ, flags=1536, stream_id=2, context=aaab18d025e0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.199.297 [stream.cpp:189]619439 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.320 [stream.cc:3161] 619439 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.199.333 [stream.cpp:195]619439 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.199.371 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab18e2db80. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.199.400 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.199.415 [stream.cpp:48]619439 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.434 [api_impl.cc:1120] 619439 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.466 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.497 [stream.cc:2246] 619439 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.534 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.551 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.787 [npu_driver.cc:6125] 619439 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.839 [npu_driver.cc:6137] 619439 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=368, streamId=3, logicCq=368, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.861 [npu_driver.cc:6189] 619439 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=3, logicCq=368. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.879 [npu_driver.cc:5916] 619439 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c40aa000, addr=dfffc40aa000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.895 [stream.cc:653] 619439 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619439,python3.9):2025-04-03-09:35:41.199.940 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.199.959 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.200.277 [stream.cc:291] 619439 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.200.894 [stream.cc:688] 619439 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=5, streamResId=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.200.914 [stream_state_callback_manager.cc:45] 619439 Notify: not reg stream state callback func! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.200.927 [api_impl.cc:1162] 619439 StreamCreate: Succ, flags=1536, stream_id=3, context=aaab18d025e0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.200.943 [stream.cpp:189]619439 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.200.964 [stream.cc:3161] 619439 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.200.979 [stream.cpp:195]619439 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.201.009 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab18f5e630. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.104 [callback.cpp:25]619439 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281470983466912. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.129 [npu_driver.cc:462] 619439 EventIdAlloc: event id alloc begin, device_id=5, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.148 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.192 [npu_driver.cc:483] 619439 EventIdAlloc: Event id alloc success, device_id=5, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.198 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.209 [event.cc:347] 619439 GenEventId: Event id alloc success, device_id=5, tsId=0, eventFlag=1, event_id=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.216 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.231 [subscribe.cc:54] 619439 AssignGroupId: give groupId=832 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.246 [npu_driver.cc:3775] 619439 SqCqAllocate: deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.269 [npu_driver.cc:3785] 619439 SqCqAllocate: alloc success, deviceId=5 sqId=120, cqId=120. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.285 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.298 [subscribe.cc:131] 619439 Insert: callback subscribe: groupId=832, threadId=281470983466912, devId=5, tsId=0, streamId=2, sqId=120, cqId=120 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.314 [callback.cpp:31]619439 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281470983466912. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.321 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.201.340 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab18e2db80. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.340 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.416 [callback.cpp:25]619439 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281470975012768. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.438 [npu_driver.cc:462] 619439 EventIdAlloc: event id alloc begin, device_id=5, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.458 [npu_driver.cc:483] 619439 EventIdAlloc: Event id alloc success, device_id=5, ts_id=0, event_id=1, eventFlag=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.456 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.473 [event.cc:347] 619439 GenEventId: Event id alloc success, device_id=5, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.490 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.494 [subscribe.cc:54] 619439 AssignGroupId: give groupId=833 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.505 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.511 [npu_driver.cc:3775] 619439 SqCqAllocate: deviceId=5, tsId=0, groupId=833. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.519 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.528 [npu_driver.cc:3785] 619439 SqCqAllocate: alloc success, deviceId=5 sqId=121, cqId=121. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.532 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.543 [subscribe.cc:131] 619439 Insert: callback subscribe: groupId=833, threadId=281470975012768, devId=5, tsId=0, streamId=3, sqId=121, cqId=121 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.201.563 [callback.cpp:31]619439 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281470975012768. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.201.569 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.201.590 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab18f5e630. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.201.610 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.201.699 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.202.037 [npu_driver.cc:2616] 619439 MemGetInfoEx: memory info type=1, type=2, free=8863059968, hug_free=56245616640 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.081 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.161 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.204 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.231 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.253 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62092M, Reserved MOC size for Other Components(HCCL/rts/etc.):3892M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58200M. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.202.278 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58200M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.202.307 [event.cpp:260]619439 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.202.329 [api_impl.cc:5713] 619439 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.406 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.428 [timeout_set_task.cc:115] 619439 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.442 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.459 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.481 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.496 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.514 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.527 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.560 [stars_engine.cc:622] 619439 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=366, retCode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.577 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.591 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.617 [stream.cc:2886] 619439 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.635 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.650 [stream.cc:2886] 619439 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.665 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.680 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.694 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.709 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.729 [stars_engine.cc:437] 619439 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.746 [stream.cc:2886] 619439 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.203.759 [event.cpp:262]619439 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.203.777 [event.cpp:269]619439 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.797 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.814 [timeout_set_task.cc:115] 619439 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.830 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.844 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.862 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.876 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.892 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.908 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.925 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.939 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.954 [stream.cc:2886] 619439 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.968 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.983 [stream.cc:2886] 619439 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.203.998 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.013 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.040 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.051 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.069 [stars_engine.cc:437] 619439 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.087 [stream.cc:2886] 619439 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.099 [event.cpp:277]619439 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.126 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.168 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.190 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 5 has been set [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.208 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.226 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.274 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.306 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:41.204.354 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.479 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.500 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.534 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.557 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.204.575 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.588 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.757 [acl.cpp:288]619439 aclInit: start to execute aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.794 [acl.cpp:298]619439 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.204.910 [acl.cpp:242]619439 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.213.975 [error_manager.cc:673]619439 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.214.721 [acl.cpp:315]619439 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:41.214.802 [prof_atls_plugin.cpp:210] (tid:619439) Module[45] register callback of ctrl handle. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.214.853 [ge_executor.cc:273]619439 Initialize:Init GeExecutor begin. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.214.922 [op_lib_register.cc:117]619439 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.012 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.049 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.083 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.117 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.147 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.179 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.210 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.240 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.271 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.301 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.343 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.374 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.405 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.435 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.466 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.496 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.526 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.557 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.587 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.215.617 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.652 [op_lib_register.cc:165]619439 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.697 [plugin_manager.cc:616]619439 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.713 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.742 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.766 [plugin_manager.cc:624]619439 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.215.782 [plugin_manager.cc:206]619439 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.216.662 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.216.706 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.216.725 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.216.742 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.216.789 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.134 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.168 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.185 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.198 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.234 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.275 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.301 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.316 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.333 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.349 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.683 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.718 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.735 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.217.748 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.217.784 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.120 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.153 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.170 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.182 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.216 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.253 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.269 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.283 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.300 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.331 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.664 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.696 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.712 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.218.727 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.218.763 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.095 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.127 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.144 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.157 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.194 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.231 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.246 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.261 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.278 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.292 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.626 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.658 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.686 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.219.699 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.219.733 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.066 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.099 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.116 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.128 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.171 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.209 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.226 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.241 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.255 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.269 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.602 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.634 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.649 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.220.663 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.220.697 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.042 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.074 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.091 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.104 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.139 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.176 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.192 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.205 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.220 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.236 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.252 [plugin_manager.cc:218]619439 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.274 [plugin_manager.cc:422]619439 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.289 [plugin_manager.cc:155]619439 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.312 [plugin_manager.cc:158]619439 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.328 [plugin_manager.cc:426]619439 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.649 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.692 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.709 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.221.724 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.763 [plugin_manager.cc:439]619439 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.785 [plugin_manager.cc:175]619439 ReversePathString:Enter ReversePathString schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.221.806 [plugin_manager.cc:187]619439 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.222.123 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.222.153 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.222.174 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.222.222 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619439,python3.9):2025-04-03-09:35:41.237.366 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.237.954 [op_tiling_registry.cc:162]619439 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.071 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.179 [op_tiling_registry.cc:162]619439 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.280 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.379 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.474 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.582 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.602 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.624 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.717 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.810 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.901 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619439,python3.9):2025-04-03-09:35:41.238.995 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.092 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.413 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.501 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.603 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.704 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.800 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.239.903 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.006 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.194 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.294 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.401 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.499 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.518 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.632 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.740 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.760 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.850 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.240.947 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.046 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.236 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.403 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.497 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.596 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619439,python3.9):2025-04-03-09:35:41.241.698 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.117 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.221 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.326 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.502 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.608 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.723 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.824 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.242.929 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.024 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.123 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.231 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.329 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.358 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.457 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.558 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.656 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.674 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.767 [op_tiling_registry.cc:122]619439 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.862 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619439,python3.9):2025-04-03-09:35:41.243.964 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.061 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.183 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.369 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.550 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.624 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.696 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.869 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.955 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.244.982 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.007 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.033 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.155 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.228 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.316 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.387 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.462 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.543 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.569 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.640 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.666 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.691 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.766 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.846 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.245.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.072 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.143 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.223 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.295 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.371 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.442 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.615 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.695 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.845 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.246.928 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.247.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.250.914 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619439,python3.9):2025-04-03-09:35:41.250.944 [op_tiling_registry.cc:148]619439 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619439,python3.9):2025-04-03-09:35:41.252.322 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.252.451 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.252.570 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.252.835 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.252.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.065 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.434 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.584 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.731 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.844 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.253.952 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.075 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.192 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.308 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.337 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.485 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.603 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.744 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.861 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.254.972 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.109 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.370 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.488 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.596 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.713 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.841 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.255.952 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.068 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.198 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.311 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.427 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.668 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.784 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.256.926 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.135 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.250 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.613 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.748 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.863 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.257.894 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.109 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.136 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.253 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.467 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.578 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.777 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.806 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.258.911 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.020 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.209 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.315 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.436 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.545 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.656 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.259.848 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.010 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.336 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.461 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.576 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.691 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.799 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.260.912 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.170 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.382 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffef1345d70, registered count:1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.549 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.572 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffef134e248, registered count:2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.695 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.808 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.261.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.062 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.178 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.292 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.629 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.734 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.773 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.888 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.262.992 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.109 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.217 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.322 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.441 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.656 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.801 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.829 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.855 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.263.880 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.029 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.142 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.261 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.375 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.587 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.614 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.639 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.756 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.863 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.264.979 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.170 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.282 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.388 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.494 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.617 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.724 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.838 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.265.974 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.389 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.500 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.605 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.829 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.266.942 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.267.053 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.267.160 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.267.357 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.267.473 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.267.589 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.042 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.244 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.362 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.473 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.691 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.799 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.270.912 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.016 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.133 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.164 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.290 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.432 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.461 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.564 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.670 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.781 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.271.890 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.558 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.272.905 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.273.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.274.326 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.274.642 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.275.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.275.662 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.275.784 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.275.898 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.276.044 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.276.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.276.451 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.278.599 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.279.409 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.279.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.279.723 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.280.696 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.280.849 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.575 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.809 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.281.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.039 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.375 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.488 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.612 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.763 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.877 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.282.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.290 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.399 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.516 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.664 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.283.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.073 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.102 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.296 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.412 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.543 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.718 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.832 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.284.994 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.105 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.225 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.341 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.482 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.617 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.755 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.875 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.285.986 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.104 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.397 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.593 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.736 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.286.906 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.087 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.262 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.570 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.691 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.802 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.911 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.287.939 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.125 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.251 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.368 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.484 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.596 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.706 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.813 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.841 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.288.958 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.289.188 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.290.438 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.292.836 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.293.039 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.293.242 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.293.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.293.710 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.293.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.294.688 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.294.847 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.294.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.121 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.274 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.414 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.564 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.722 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.295.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.131 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.236 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.541 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.572 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.602 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.627 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.652 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.678 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.703 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.729 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.778 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.803 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.844 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.869 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.898 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.923 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.947 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.296.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.021 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.046 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.094 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.119 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.143 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.167 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.194 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.219 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.242 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.291 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.315 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.366 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.390 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.424 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.449 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.497 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.524 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.552 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.576 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.601 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.624 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.649 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.674 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.723 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.747 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.771 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.795 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.845 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.869 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.893 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.917 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.297.941 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.437 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.466 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.492 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.544 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.690 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.804 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.298.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.028 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.145 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.257 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.427 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.708 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.833 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.299.985 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.011 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.131 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.168 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.199 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.309 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.805 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.300.923 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.234 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.431 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.549 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.665 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.693 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.719 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.301.896 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.063 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.215 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.556 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.708 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.827 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.302.987 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.157 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.341 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.602 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.845 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.303.961 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.122 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.406 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.530 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.776 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.304.890 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.305.336 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.305.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.305.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.031 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.304 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.420 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.656 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.784 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.306.912 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.037 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.198 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.441 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.632 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.749 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.878 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.307.994 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.214 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.397 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.634 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.756 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.308.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.000 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.118 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.231 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.670 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.309.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.310.124 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.310.312 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.310.495 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.310.716 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.310.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.164 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.214 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.237 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.297 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.322 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.311.347 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.312.100 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.312.529 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.312.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.313.148 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.313.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.313.766 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.313.905 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.314.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.316.830 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.318.641 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.318.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.318.928 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.319.045 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.319.337 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.319.501 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.319.688 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.027 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.060 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.087 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.348 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.383 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.550 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.730 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.320.898 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.461 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.637 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.813 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.321.999 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.176 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.342 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.371 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.518 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.712 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.747 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.858 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619439,python3.9):2025-04-03-09:35:41.322.891 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.045 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.176 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.452 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.579 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.706 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.817 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.323.930 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.324.129 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.324.359 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.324.736 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.325.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.325.347 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.325.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.325.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.057 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.162 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.197 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.304 [op_tiling_registry.cc:134]619439 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.335 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.326.573 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.327.689 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.144 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.638 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.814 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.328.961 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.110 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.141 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.465 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.618 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.329.946 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.330.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.330.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.330.965 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.331.159 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.332.239 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.332.690 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.333.253 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.333.424 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.333.587 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.333.710 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.333.879 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.066 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.182 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.211 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.552 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.669 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.831 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.334.992 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.335.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.335.436 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.335.631 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.335.785 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.335.906 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.024 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.209 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.572 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.606 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.800 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.336.967 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.337.152 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.338.661 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.344.347 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.344.396 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.347.181 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.347.452 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.347.749 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.348.078 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.348.300 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.348.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.348.681 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.348.838 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.016 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.244 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.496 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.625 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.349.775 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.237 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.394 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.551 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.584 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.699 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.350.828 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.006 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.203 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.398 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.560 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.683 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.802 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.351.830 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.352.598 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.352.817 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.352.996 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.117 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.302 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.435 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.550 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.578 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.605 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.353.951 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.040 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.281 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.476 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.507 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.731 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.354.763 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.511 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.607 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.635 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.705 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.783 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.807 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.357.888 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.044 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.065 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.178 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.199 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.220 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.240 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.408 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.358.429 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.359.611 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffef1c98a10, registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.359.646 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffef1c98c70, registered count:1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.360.067 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.360.552 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.362.227 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.362.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.362.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.362.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.363.317 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.393.713 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.393.851 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.394.039 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.394.121 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.394.270 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.394.406 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.028 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.119 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.368 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.450 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.395.844 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.396.011 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.396.425 [op_tiling_manager.cc:79]619439 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.504 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.721 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.768 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.832 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.893 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.934 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.396.993 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.397.041 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.034 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.072 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.097 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.118 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.142 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.476 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff200fd9b0, registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.501 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.525 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.546 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.565 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.879 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff20100da0, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.904 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.924 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.944 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.398.960 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.399.271 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.399.328 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.399.418 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.194 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff200b72c0, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.231 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.254 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.273 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.292 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.625 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff200bb978, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.650 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.668 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.687 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.400.704 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.401.002 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.055 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.131 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.197 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.239 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.285 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.401.320 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.360 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.401 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.401.434 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.472 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.510 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.548 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.594 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.635 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.401.667 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.704 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.748 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.401.781 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.818 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.856 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.894 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.931 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.401.970 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.008 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.047 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.085 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.118 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.155 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.192 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.230 [op_tiling_manager.cc:73]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.267 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.305 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.336 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.381 [op_tiling_manager.cc:58]619439 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.402.420 [op_tiling_manager.cc:67]619439 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.453 [op_tiling_manager.cc:75]619439 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.471 [op_tiling_manager.cc:109]619439 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [186775] micro seconds. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.606 [plugin_manager.cc:639]619439 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.625 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.653 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.402.718 [host_cpu_engine.cc:254]619439 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.408.130 [host_cpu_engine.cc:273]619439 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619439,python3.9):2025-04-03-09:35:41.408.214 [host_cpu_engine.cc:254]619439 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.414.359 [host_cpu_engine.cc:267]619439 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619439,python3.9):2025-04-03-09:35:41.415.137 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619439] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.173 [folding.cc:404][InitCpuConstantFoldingNew][tid:619439]:"Init cpu constant folding begin." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.208 [folding.cc:408][InitCpuConstantFoldingNew][tid:619439]:"Number of registered ops: 234" [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.226 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Acos]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.245 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Acosh]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.262 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.279 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.292 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.308 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.321 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.351 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Add]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.363 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Addcdiv]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.377 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Addcmul]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.393 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[AffineGrid]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.406 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ApplyAdamV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.420 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Asin]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.436 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Asinh]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.448 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Assert]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.463 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Atan]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.478 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Atan2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.490 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Atanh]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[AvgPoolV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.520 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Betainc]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Bucketize]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.547 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CTCLossV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.563 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.575 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CacheSwapTable]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.589 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.603 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CaseCondition]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.618 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cast]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.631 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Ceil]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.645 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CheckNumerics]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.660 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ClipByValue]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.673 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ClipByValueV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.687 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Combinations]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.702 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CompareAndBitpack]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.715 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.730 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ConcatV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.745 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Conj]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.766 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ConjugateTranspose]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.779 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.793 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cosh]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.808 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[CropAndResize]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.821 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cross]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.834 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cummax]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.850 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cummin]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.863 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Cumsum]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.876 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Div]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.892 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[DynamicStitch]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.905 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EditDistance]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.919 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Elu]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.935 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EluGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.948 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.962 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.978 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.415.990 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.004 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.018 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.034 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.046 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.060 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.076 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.088 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.103 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.118 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.130 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.145 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingLookup]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.174 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.197 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.213 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.241 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.259 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Equal]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.271 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Expand]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ExpandDims]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.301 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Exponential]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.313 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.327 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Fill]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.342 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[FillDiagonal]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Fills]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.369 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Floor]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.385 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[FloorDiv]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[FloorMod]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.411 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GEMM]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.425 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GatherElements]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.439 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GatherV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.453 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Geometric]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.466 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GetDynamicDims]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.480 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Greater]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.495 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GreaterEqual]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.508 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GridSampler2D]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.522 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.537 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GridSampler3D]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.555 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.568 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Histogram]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.588 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.601 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Identity]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.615 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[IndexToAddr]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.630 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InitPartitionMap]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.658 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.673 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.686 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InvGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.699 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[InvertPermutation]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.714 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[IsClose]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.727 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[IsInf]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.742 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[IsNan]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.757 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Less]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.770 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[LinSpace]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.784 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[LinearSumAssignment]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.799 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Log]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.813 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Log1p]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.827 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.841 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.854 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.868 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MaskedFill]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.883 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MaskedScatter]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.896 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MaskedSelect]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.910 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.925 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MatMul]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.938 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.953 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MatrixDiagV3]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.968 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MinAreaPolygons]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.416.985 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Mul]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.014 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.029 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.045 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.058 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NLLLoss]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.074 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NLLLossGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.091 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NMSWithMask]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.107 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Neg]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.122 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.135 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NonZero]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.151 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.166 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.179 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[NotEqual]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.196 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[OnesLike]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.208 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[PSAMask]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.226 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[PSAMaskGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.240 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Pad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.256 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[PadD]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.269 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[PadV3]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.282 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[PadV3Grad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.299 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Pinverse]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.316 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Poisson]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.329 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Pow]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.346 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.364 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RNNTLoss]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.377 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RandomStandardNormal]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.393 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RandomUniform]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.406 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RealDiv]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.423 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ReduceAll]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.444 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ReduceMean]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.458 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ReduceSum]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.472 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Reshape]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.487 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Resize]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.500 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ResizeBilinear]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.515 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.530 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.543 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ResizeGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.559 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ResizeTrilinear]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.574 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ReverseSequence]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.587 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RightShift]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.601 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.616 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[RotatedNMS]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.634 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Round]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.647 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Rsqrt]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.662 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[STFT]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.680 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.693 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ScatterElements]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.707 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ScatterNdAdd]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.720 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ScatterNdMax]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.733 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ScatterNdMin]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.747 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SearchSorted]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.763 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SelectV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.776 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Sigmoid]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.790 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SigmoidGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.806 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Sign]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.819 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SilentCheck]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.832 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SilentCheckV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.848 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SilentCheckV3]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.868 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SliceWrite]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.880 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SoftmaxV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.899 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Sort]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.914 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.927 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SparseSegmentMean]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.942 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.957 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SparseSegmentSum]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.972 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SparseToDense]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.417.988 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SpatialTransformer]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.001 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SplitD]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.015 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SplitV]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.030 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Sqrt]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.042 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SqrtGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.056 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Square]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.072 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[SquaredDifference]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.084 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[StatelessBernoulli]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.097 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.111 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[StatelessRandperm]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.125 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[StridedSlice]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.141 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[StridedSliceV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.154 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Sub]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.168 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TableToResource]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.183 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Tan]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.196 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TanhGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.210 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TensorEqual]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.225 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.239 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ThreeNN]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.254 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Tile]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.274 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TileWithAxis]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.288 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TopK]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.302 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TopKPQDistance]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.318 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.331 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TopKV2D]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.345 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Trace]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.361 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TransData]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.374 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[TransDataRNN]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.388 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Tril]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.403 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Triu]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.417 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.435 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.451 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UninitPartitionMap]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.464 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UniqueConsecutive]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.494 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.507 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.521 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.534 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.550 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.563 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ViewCopy]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.577 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Voxelization]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.593 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[WarpAffine]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.605 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[ZerosLike]." [INFO] AICPU(619439,python3.9):2025-04-03-09:35:41.418.619 [folding.cc:413][InitCpuConstantFoldingNew][tid:619439]:"Register op[Zeta]." [INFO] OP(619439,python3.9):2025-04-03-09:35:41.418.639 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619439] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619439,python3.9):2025-04-03-09:35:41.418.655 [host_cpu_engine.cc:273]619439 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619439,python3.9):2025-04-03-09:35:41.418.789 [plugin_manager.cc:872]619439 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.422.209 [plugin_manager.cc:883]619439 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.280 [ops_kernel_executor_manager.cc:37]619439 Initialize:No library to load [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.300 [plugin_manager.cc:540]619439 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.317 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.345 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.366 [plugin_manager.cc:547]619439 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.422.383 [plugin_manager.cc:206]619439 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.195 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.239 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.256 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.269 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.315 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.659 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.695 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.711 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.726 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.764 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.816 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.833 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.847 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.864 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.896 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.911 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.423.924 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.423.938 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.283 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.320 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.335 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.348 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.384 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.719 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.753 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.770 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.782 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.820 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.868 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.885 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.897 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.913 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.942 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.958 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.424.969 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.424.983 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.425.314 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.347 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.363 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.376 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.425.411 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.425.746 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.780 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.796 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.808 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.425.843 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.890 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.906 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.917 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.425.932 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.964 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.978 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.425.991 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.004 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.333 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.367 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.383 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.395 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.430 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.763 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.796 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.812 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.824 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.871 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.906 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.922 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.935 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.426.954 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.984 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.426.999 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.012 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.027 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.356 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.388 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.405 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.417 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.462 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.797 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.830 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.846 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.859 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.906 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.942 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.959 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.427.971 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.427.985 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.017 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.032 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.044 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.058 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.072 [plugin_manager.cc:218]619439 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.091 [plugin_manager.cc:422]619439 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.109 [plugin_manager.cc:155]619439 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.131 [plugin_manager.cc:158]619439 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.147 [plugin_manager.cc:426]619439 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.504 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.540 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.556 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.428.570 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.607 [plugin_manager.cc:439]619439 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.629 [ge_executor.cc:259]619439 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619439,python3.9):2025-04-03-09:35:41.428.982 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.016 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.038 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.144 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.162 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.239 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.265 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.429.295 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.344 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.360 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.429.389 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.435 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.451 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.517 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.533 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.596 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.611 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.429.639 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.683 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.698 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.429.725 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.768 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.783 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.846 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.867 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.931 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.429.946 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.429.974 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.017 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.032 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.058 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.101 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.116 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.179 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.194 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.257 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.272 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.298 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.341 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.356 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.382 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.432 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.448 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.511 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.526 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.589 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.605 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.631 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.675 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.689 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.715 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.757 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.773 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.798 [opsproto_manager.cc:123]619439 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.839 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.902 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.918 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.930 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.430.942 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.430.958 [opsproto_manager.cc:147]619439 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.431.279 [operator_factory_impl.cc:215]619439 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619439,python3.9):2025-04-03-09:35:41.431.305 [operator_factory_impl.cc:269]619439 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619439,python3.9):2025-04-03-09:35:41.431.341 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.437.446 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.245 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.330 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.375 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.409 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.440 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.469 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.497 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.525 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.438.554 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.457.996 [operator_factory_impl.cc:368]619439 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619439,python3.9):2025-04-03-09:35:41.458.234 [operator_factory_impl.cc:284]619439 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.429 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.454 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.482 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.496 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.510 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619439,python3.9):2025-04-03-09:35:41.458.533 [operator_factory_impl.cc:254]619439 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619439,python3.9):2025-04-03-09:35:41.458.557 [operator_factory_impl.cc:340]619439 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619439,python3.9):2025-04-03-09:35:41.458.586 [operator_factory_impl.cc:307]619439 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.615 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.640 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.660 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.458.777 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.642 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.675 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.694 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.718 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.739 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.756 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.778 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.817 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.834 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.851 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.869 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.896 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.912 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.941 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.973 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.459.994 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.062 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.115 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.135 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.153 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.197 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.460.220 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.449 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.480 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.502 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.522 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.538 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.556 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.579 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.617 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.632 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.648 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.743 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.761 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.777 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.793 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.806 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.461.822 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.152 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.173 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.191 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.205 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.232 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.245 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.263 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.288 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.305 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.320 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.336 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.358 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.375 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.607 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.629 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.687 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.704 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.717 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.733 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.462.747 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.678 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.708 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.727 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.748 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.765 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.783 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.800 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.463.923 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.012 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.424 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.448 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.465 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.479 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.494 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.506 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.464.987 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.310 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.336 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.353 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.370 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.539 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.689 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.465.986 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.011 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.051 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.068 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.663 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.707 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.727 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.750 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.769 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.788 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.813 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.835 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.897 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.466.937 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.467.354 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.467.549 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.467.833 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.467.863 [operator_factory_impl.cc:222]619439 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.487 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.529 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.556 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.579 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.634 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.684 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.708 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.733 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.758 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.784 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.808 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.832 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.856 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.886 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.911 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.468.990 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.011 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.034 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.055 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.076 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.126 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.150 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.174 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.234 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.258 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.303 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.352 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.377 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.424 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.447 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.495 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.543 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.571 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.604 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.628 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.652 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.675 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.700 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.723 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.747 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.768 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.788 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.813 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.838 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.888 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.939 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.965 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.469.988 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.039 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.063 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.088 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.136 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.160 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.214 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.243 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.292 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.317 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.342 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.366 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.416 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.445 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.469 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.494 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.515 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.561 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.584 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.614 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.639 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.662 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.685 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.710 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.734 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.759 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.783 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.809 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.837 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.861 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.885 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.909 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.936 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.960 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.470.984 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.008 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.056 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.103 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.127 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.151 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.198 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.247 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.273 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.297 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.396 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.420 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.449 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.473 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.496 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.517 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.538 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.562 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.588 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.609 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.629 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.699 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.722 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.746 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.818 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.839 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.863 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.908 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.928 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.948 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.969 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.471.989 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.016 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.141 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.217 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.242 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.268 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.292 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.317 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.343 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.414 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.438 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.462 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.511 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.535 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.559 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.585 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.610 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.634 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.658 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.681 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.705 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.729 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.760 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.785 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.809 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.836 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.861 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.885 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.909 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.934 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.958 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.472.983 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.008 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.057 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.107 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.131 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.158 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.182 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.206 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.230 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.254 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.306 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.329 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.354 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.383 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.407 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.431 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.461 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.489 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.513 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.559 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.607 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.631 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.655 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.679 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.704 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.749 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.772 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.797 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.821 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.845 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.873 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.900 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.921 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.944 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.974 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.473.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.022 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.139 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.249 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.353 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.587 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.613 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.645 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.667 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.788 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.474.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.116 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.216 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.341 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.485 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.631 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.742 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.767 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.800 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.822 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.868 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.475.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.006 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.250 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.358 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.384 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.609 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.633 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.673 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.778 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.476.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.000 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.103 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.127 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.229 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.335 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.362 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.479 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.591 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.808 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.833 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.477.939 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.162 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.266 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.374 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.479 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.509 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.538 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.564 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.586 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.611 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.636 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.687 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.714 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.738 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.764 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.478.982 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.005 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.026 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.068 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.089 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.203 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.306 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.410 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.433 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.642 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.670 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.692 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.812 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.479.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.028 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.135 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.256 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.366 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.581 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.694 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.796 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.938 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.480.960 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.199 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.307 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.412 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.513 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.621 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.730 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.867 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.481.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.183 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.206 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.310 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.421 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.524 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.627 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.743 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.849 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.875 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.482.898 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.038 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.100 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.120 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.160 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.181 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.201 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.218 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.237 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.257 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.277 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.299 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.320 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.446 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.549 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.664 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.766 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.483.989 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.097 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.126 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.151 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.291 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.343 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.500 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.539 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.642 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.753 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.856 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.484.959 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.161 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.261 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.413 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.520 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.639 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.740 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.853 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.966 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.485.993 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.018 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.044 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.144 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.247 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.356 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.560 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.774 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.878 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.486.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.104 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.221 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.325 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.434 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.462 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.565 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.692 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.717 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.739 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.765 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.786 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.811 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.833 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.854 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.957 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.487.988 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.096 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.309 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.418 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.521 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.644 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.747 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.771 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.870 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.488.979 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.186 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.295 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.412 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.523 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.550 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.575 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.601 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.626 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.674 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.722 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.745 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.769 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.796 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.820 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.867 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.891 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.914 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.940 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.489.964 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.178 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.205 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.433 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.542 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.616 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.719 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.829 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.490.940 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.128 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.232 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.337 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.442 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.554 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.663 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.774 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.879 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.491.903 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.005 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.230 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.337 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.448 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.571 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.689 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.800 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.492.901 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.229 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.440 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.544 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.644 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.753 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.860 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.493.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.205 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.320 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.426 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.536 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.649 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.757 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.858 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.494.963 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.429 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.535 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.637 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.745 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.495.859 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.024 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.261 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.371 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.485 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.596 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.715 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.818 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.496.920 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.027 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.050 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.163 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.268 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.375 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.409 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.451 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.496 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.516 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.536 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.556 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.678 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.796 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.497.907 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.011 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.220 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.244 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.495 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.605 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.715 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.822 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.928 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.498.994 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.102 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.127 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.149 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.256 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.299 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.403 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.425 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.446 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.467 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.668 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.774 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.798 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.922 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.499.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.067 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.179 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.282 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.307 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.427 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.449 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.471 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.574 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.684 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.791 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.500.897 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.103 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.204 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.317 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.426 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.651 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.755 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.501.983 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.091 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.198 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.347 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.372 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.495 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.606 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.710 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.817 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.502.929 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.033 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.149 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.253 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.473 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.498 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.519 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.785 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.503.900 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.010 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.122 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.247 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.356 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.578 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.683 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.785 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.504.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.109 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.215 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.315 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.426 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.528 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.638 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.743 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.848 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.505.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.055 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.164 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.505 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.617 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.724 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.506.837 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.057 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.174 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.290 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.404 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.611 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.717 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.824 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.507.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.041 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.144 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.259 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.466 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.695 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.804 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.508.908 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.022 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.562 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.772 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.509.896 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.034 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.104 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.138 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.169 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.200 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.227 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.253 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.279 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.300 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.342 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.395 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.418 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.531 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.653 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.762 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.866 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.510.970 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.091 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.197 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.428 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.640 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.757 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.862 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.964 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.511.995 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.017 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.038 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.210 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.276 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.352 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.386 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.415 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.444 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.499 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.559 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.581 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.616 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.651 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.679 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.707 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.732 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.809 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.512.953 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.067 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.174 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.290 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.499 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.609 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.634 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.745 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.855 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.513.961 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.063 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.088 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.191 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.299 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.401 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.504 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.582 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.664 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.882 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.514.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.107 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.219 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.296 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.395 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.492 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.606 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.713 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.515.917 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.027 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.058 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.182 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.293 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.401 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.623 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.869 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.516.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.188 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.295 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.441 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.529 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.565 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.626 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.732 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.838 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.945 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.517.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.071 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.283 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.388 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.492 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.594 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.618 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.640 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.781 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.806 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.518.887 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.005 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.109 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.221 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.326 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.666 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.772 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.855 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.519.985 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.093 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.213 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.324 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.432 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.533 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.639 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.720 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.747 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.520.974 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.199 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.438 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.555 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.584 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.610 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.636 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.699 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.725 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.748 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.769 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.790 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.813 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.837 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.863 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.887 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.521.998 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.209 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.358 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.386 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.489 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.599 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.701 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.811 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.927 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.972 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.522.993 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.128 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.233 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.489 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.514 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.626 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.758 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.861 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.523.967 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.068 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.182 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.208 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.322 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.508 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.559 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.589 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.615 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.640 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.675 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.699 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.723 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.748 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.773 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.799 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.823 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.931 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.955 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.524.976 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.090 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.118 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.145 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.198 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.247 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.271 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.525.322 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.421 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.518 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.612 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.692 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.784 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.859 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.526.976 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.186 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.262 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.416 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.440 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.579 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.659 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.873 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.527.952 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.385 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.411 [opsproto_manager.cc:162]619439 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.458 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.498 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.931 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.528.991 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.014 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.090 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.109 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.167 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.184 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.195 [opp_so_manager.cc:56]619439 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.213 [plugin_manager.cc:540]619439 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.226 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.251 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.269 [plugin_manager.cc:547]619439 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.285 [plugin_manager.cc:206]619439 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.648 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.529.687 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.529.703 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.529.716 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.529.756 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.098 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.132 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.160 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.172 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.208 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.248 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.265 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.280 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.296 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.326 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.342 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.354 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.368 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.699 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.733 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.750 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.530.762 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.530.798 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.131 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.176 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.192 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.205 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.240 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.276 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.292 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.305 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.319 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.351 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.367 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.380 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.395 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.725 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.758 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.774 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.531.786 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.531.822 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.154 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.218 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.237 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.249 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.285 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.321 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.337 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.350 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.368 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.397 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.412 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.424 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.438 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.766 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.799 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.815 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.532.828 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.532.863 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.196 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.244 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.261 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.273 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.309 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.345 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.361 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.373 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.388 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.420 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.435 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.447 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.460 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.787 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.820 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.837 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.533.848 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.533.890 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.236 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.269 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.286 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.298 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.332 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.368 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.384 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.396 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.410 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.443 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.458 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.470 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.482 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.494 [plugin_manager.cc:218]619439 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.525 [plugin_manager.cc:422]619439 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.538 [plugin_manager.cc:155]619439 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.559 [plugin_manager.cc:158]619439 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.574 [plugin_manager.cc:426]619439 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.896 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.930 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.947 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.534.959 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.534.994 [plugin_manager.cc:439]619439 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.309 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.336 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.357 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.462 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.479 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.535.492 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.537.800 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.537.842 [op_impl_registry_holder_manager.cc:216]619439 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.537.875 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.537.924 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.538.415 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.538.527 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.538.546 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.538.559 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.540.776 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.540.815 [op_impl_registry_holder_manager.cc:216]619439 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.540.836 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.540.875 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.209 [op_impl_space_registry.cc:80]619439 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.231 [op_impl_space_registry.cc:83]619439 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.251 [op_impl_space_registry.cc:80]619439 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.266 [op_impl_space_registry.cc:83]619439 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619439,python3.9):2025-04-03-09:35:41.541.280 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.326 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.541.362 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.541.460 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.541.478 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.541.490 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.542.196 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.542.225 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.542.318 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.542.336 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.542.349 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.542.939 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.542.966 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.543.006 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.543.040 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.124 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.141 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.152 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.718 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.543.744 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.844 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.862 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.543.873 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.544.450 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.544.478 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.544.520 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.544.554 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.544.637 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.544.654 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.544.666 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.545.228 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.545.254 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.545.345 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.545.362 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.545.374 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.545.927 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.545.953 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.546.009 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.546.044 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.127 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.143 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.154 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.713 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.546.739 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.826 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.843 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.546.856 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.547.415 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.547.440 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.547.479 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.547.513 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.547.544 [opp_so_manager.cc:97]619439 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.547.626 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.547.671 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.547.687 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.547.699 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.218 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.266 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.308 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.348 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.386 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.404 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.423 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.442 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.495 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.514 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.551 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.569 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.587 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.607 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.625 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.643 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.661 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.693 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.712 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.730 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.749 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.767 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.785 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.804 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.823 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.841 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.859 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.878 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.897 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.934 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.952 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.572.989 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.008 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.026 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.044 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.062 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.099 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.118 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.137 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.156 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.179 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.197 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.215 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.233 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.252 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.271 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.289 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.307 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.326 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.382 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.419 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.437 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.474 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.490 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.508 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.527 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.564 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.601 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.619 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.637 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.655 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.680 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.717 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.735 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.772 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.791 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.809 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.828 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.846 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.865 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.901 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.920 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.938 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.573.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.009 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.027 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.045 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.064 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.120 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.159 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.182 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.202 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.220 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.239 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.257 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.275 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.294 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.312 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.348 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.370 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.412 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.434 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.454 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.497 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.517 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.538 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.558 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.579 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.600 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.620 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.641 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.661 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.681 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.706 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.726 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.752 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.777 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.798 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.842 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.862 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.905 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.944 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.964 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.574.989 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.010 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.031 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.176 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.197 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.218 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.238 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.259 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.346 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.368 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.388 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.408 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.428 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.450 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.492 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.514 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.534 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.554 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.576 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.600 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.623 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.645 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.665 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.686 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.705 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.725 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.746 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.766 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.787 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.807 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.827 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.847 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.868 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.890 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.920 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.941 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.962 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.575.983 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.023 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.046 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.067 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.087 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.107 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.128 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.148 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.189 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.213 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.233 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.253 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.278 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.361 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.382 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.402 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.422 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.446 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.466 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.533 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.553 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.574 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.594 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.615 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.637 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.657 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.678 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.741 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.851 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.576.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.149 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.254 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.281 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.307 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.432 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.633 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.727 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.827 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.577.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.048 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.073 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.095 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.117 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.214 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.313 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.336 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.358 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.379 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.421 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.442 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.632 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.741 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.840 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.578.958 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.052 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.079 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.122 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.218 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.316 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.425 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.527 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.552 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.646 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.738 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.762 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.856 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.579.960 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.054 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.153 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.193 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.285 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.385 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.485 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.592 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.687 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.779 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.805 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.827 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.848 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.869 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.935 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.957 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.979 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.580.999 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.021 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.123 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.246 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.289 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.310 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.331 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.433 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.531 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.625 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.648 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.744 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.840 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.864 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.886 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.581.980 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.084 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.187 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.281 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.377 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.472 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.575 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.670 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.765 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.860 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.582.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.019 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.213 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.238 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.329 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.425 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.521 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.619 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.744 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.842 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.583.940 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.034 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.276 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.379 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.474 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.665 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.764 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.789 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.813 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.915 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.584.996 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.017 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.038 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.102 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.122 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.142 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.163 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.186 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.227 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.248 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.270 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.365 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.554 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.655 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.850 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.948 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.585.995 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.018 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.119 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.143 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.165 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.186 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.280 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.303 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.325 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.346 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.437 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.543 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.636 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.729 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.828 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.586.926 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.020 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.128 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.220 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.316 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.411 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.508 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.631 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.653 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.678 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.768 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.880 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.587.974 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.177 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.279 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.375 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.469 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.570 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.664 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.870 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.969 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.588.994 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.088 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.183 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.206 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.230 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.251 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.293 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.316 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.339 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.553 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.648 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.749 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.853 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.589.948 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.048 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.149 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.264 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.548 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.651 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.746 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.867 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.887 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.908 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.929 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.949 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.970 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.590.990 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.010 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.030 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.051 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.079 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.102 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.132 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.153 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.195 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.216 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.240 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.436 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.556 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.652 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.744 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.815 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.591.910 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.007 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.189 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.283 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.381 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.475 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.576 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.674 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.766 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.862 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.961 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.592.985 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.091 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.187 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.380 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.479 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.505 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.601 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.696 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.593.984 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.277 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.376 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.470 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.567 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.755 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.779 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.873 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.594.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.171 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.269 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.377 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.474 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.570 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.663 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.755 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.849 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.946 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.595.970 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.070 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.296 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.489 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.591 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.685 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.781 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.878 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.596.972 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.069 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.170 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.264 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.363 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.458 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.561 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.585 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.685 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.791 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.887 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.910 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.953 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.974 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.597.995 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.016 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.036 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.148 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.172 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.194 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.386 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.489 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.584 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.678 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.702 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.725 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.820 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.914 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.598.940 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.034 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.129 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.230 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.325 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.348 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.370 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.493 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.517 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.538 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.630 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.653 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.676 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.771 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.816 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.837 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.599.932 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.030 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.124 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.147 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.178 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.274 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.298 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.404 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.498 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.598 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.633 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.655 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.746 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.884 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.600.976 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.076 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.171 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.269 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.555 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.662 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.756 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.854 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.601.947 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.043 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.146 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.240 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.346 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.480 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.525 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.628 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.723 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.822 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.602.916 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.014 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.203 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.304 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.399 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.493 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.521 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.545 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.566 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.658 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.758 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.852 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.603.949 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.042 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.256 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.353 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.554 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.649 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.750 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.845 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.604.952 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.050 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.145 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.238 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.335 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.431 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.530 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.624 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.716 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.815 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.605.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.005 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.104 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.127 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.220 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.416 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.709 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.803 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.904 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.606.999 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.094 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.193 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.295 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.394 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.484 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.677 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.775 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.875 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.607.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.065 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.173 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.274 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.370 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.470 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.563 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.659 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.756 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.870 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.899 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.957 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.608.983 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.008 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.034 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.058 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.081 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.135 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.156 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.177 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.202 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.224 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.317 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.414 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.506 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.601 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.700 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.609.971 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.097 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.193 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.294 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.389 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.580 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.607 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.631 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.652 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.673 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.801 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.829 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.855 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.883 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.907 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.933 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.960 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.610.984 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.009 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.038 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.062 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.089 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.142 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.170 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.197 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.222 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.321 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.424 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.447 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.540 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.635 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.736 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.830 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.611.942 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.035 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.062 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.183 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.282 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.385 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.478 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.501 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.597 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.691 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.787 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.885 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.612.955 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.029 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.133 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.229 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.334 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.433 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.526 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.599 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.673 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.761 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.856 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.613.954 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.049 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.241 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.364 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.456 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.555 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.648 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.743 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.614.942 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.036 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.136 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.233 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.328 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.436 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.463 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.508 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.530 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.575 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.670 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.761 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.831 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.855 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.877 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.615.969 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.286 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.385 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.478 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.501 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.523 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.686 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.756 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.857 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.616.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.051 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.145 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.349 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.443 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.536 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.605 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.629 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.617.923 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.018 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.117 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.220 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.390 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.415 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.606 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.698 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.889 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.911 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.934 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.956 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.977 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.618.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.020 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.040 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.101 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.125 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.147 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.167 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.258 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.453 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.514 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.609 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.702 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.797 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.619.901 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.025 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.049 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.071 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.095 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.199 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.294 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.392 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.487 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.511 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.630 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.653 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.748 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.842 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.620.940 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.038 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.139 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.164 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.260 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.287 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.417 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.439 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.460 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.482 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.523 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.566 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.588 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.609 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.629 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.692 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.783 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.807 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.828 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.923 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.946 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.967 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.621.988 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.009 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.029 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.049 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.078 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.099 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.119 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.235 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.314 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.503 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.572 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.646 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.622.943 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.017 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.084 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.161 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.238 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.305 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.373 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.479 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.574 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.740 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.815 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.881 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.623.901 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.624.142 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.632.890 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.637.214 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.658.428 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.658.771 [plugin_manager.cc:554]619439 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619439,python3.9):2025-04-03-09:35:41.658.795 [plugin_manager.cc:134]619439 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.658.838 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.658.856 [plugin_manager.cc:145]619439 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.658.867 [plugin_manager.cc:557]619439 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.658.882 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.658.909 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.524 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.575 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.599 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.676 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.694 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.749 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.766 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.777 [opp_so_manager.cc:66]619439 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.792 [plugin_manager.cc:616]619439 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.806 [plugin_manager.cc:108]619439 GetOppPath:Enter get opp path schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.827 [plugin_manager.cc:117]619439 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.846 [plugin_manager.cc:624]619439 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.659.862 [plugin_manager.cc:206]619439 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.267 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.310 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.328 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.341 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.386 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.730 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.765 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.781 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.795 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.835 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.875 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.892 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.660.905 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.922 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.660.938 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.273 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.307 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.335 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.348 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.383 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.721 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.754 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.770 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.783 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.819 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.856 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.872 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.661.884 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.898 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.661.913 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.246 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.279 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.295 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.307 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.345 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.695 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.728 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.744 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.759 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.795 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.832 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.848 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.662.860 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.878 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.662.894 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.227 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.259 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.276 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.288 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.322 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.657 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.690 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.706 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.718 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.766 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.802 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.818 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.663.832 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.849 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.663.863 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.215 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.251 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.269 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.282 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.318 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.653 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.686 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.702 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.714 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.749 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.786 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.813 [plugin_manager.cc:1026]619439 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.664.826 [plugin_manager.cc:322]619439 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.840 [plugin_manager.cc:212]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.856 [plugin_manager.cc:215]619439 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.871 [plugin_manager.cc:218]619439 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.893 [plugin_manager.cc:422]619439 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.907 [plugin_manager.cc:155]619439 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.930 [plugin_manager.cc:158]619439 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.664.945 [plugin_manager.cc:426]619439 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.268 [plugin_manager.cc:250]619439 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.665.302 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.665.319 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.665.331 [plugin_manager.cc:1063]619439 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.368 [plugin_manager.cc:439]619439 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.694 [plugin_manager.cc:996]619439 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.723 [plugin_manager.cc:1011]619439 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.744 [plugin_manager.cc:1015]619439 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.665.797 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.902 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.918 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.665.930 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.459 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff101b62c0, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.499 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.520 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.540 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.559 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.855 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff101ba978, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.879 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.898 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.916 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.667.931 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.201 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.668.241 [op_impl_registry_holder_manager.cc:216]619439 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.260 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.299 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.516 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.668.568 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.680 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.698 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.668.710 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.165 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.202 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.220 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.238 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.257 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.535 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff101679b0, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.560 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.579 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.597 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.628 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.904 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff1016ada0, registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.927 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.946 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.964 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.670.982 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.671.223 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.248 [op_impl_registry_holder_manager.cc:216]619439 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.671.266 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.671.302 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.614 [op_impl_space_registry.cc:84]619439 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.634 [op_impl_space_registry.cc:85]619439 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.646 [op_impl_space_registry.cc:86]619439 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.657 [op_impl_space_registry.cc:87]619439 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.670 [op_impl_space_registry.cc:93]619439 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.691 [op_impl_space_registry.cc:84]619439 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.706 [op_impl_space_registry.cc:85]619439 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.716 [op_impl_space_registry.cc:86]619439 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.729 [op_impl_space_registry.cc:87]619439 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.741 [op_impl_space_registry.cc:93]619439 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619439,python3.9):2025-04-03-09:35:41.671.755 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.817 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.857 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.671.894 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.671.994 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.012 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.024 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.559 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.672.587 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.672.634 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.732 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.749 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.672.760 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.673.368 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.673.396 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.673.455 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.673.492 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.673.528 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.673.623 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.673.640 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.673.652 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.674.073 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.098 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.140 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.674.235 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.674.252 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.674.262 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.674.811 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.838 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.882 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.931 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.674.969 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.062 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.079 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.091 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.506 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.675.531 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.675.574 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.667 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.684 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.675.695 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.676.256 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.676.284 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.676.329 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.676.366 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.676.413 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.676.507 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.676.524 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.676.536 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.676.951 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.676.974 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.019 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.113 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.129 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.141 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.684 [op_impl_space_registry.cc:240]619439 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.711 [opp_so_manager.cc:48]619439 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.756 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.792 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.826 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.865 [opp_so_manager.cc:126]619439 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:41.677.896 [opp_so_manager.cc:123]619439 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.977 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.677.993 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.678.039 [plugin_manager.cc:1126]619439 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.678.055 [plugin_manager.cc:1130]619439 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.678.066 [opp_so_manager.cc:45]619439 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.693.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.694.540 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.694.597 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.694.625 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.697.243 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619439,python3.9):2025-04-03-09:35:41.728.204 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:41.728.203 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:41.728.241 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:41.728.250 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.263 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.266 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.286 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.307 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.728.329 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.728.329 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.728.353 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:41.728.371 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.389 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:41.728.390 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] GE(619439,python3.9):2025-04-03-09:35:41.795.496 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.001 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.086 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.192 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.334 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.403 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.426 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.488 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.530 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.657 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.727 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.794 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.857 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.796.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.012 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.075 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.102 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.147 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.216 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.280 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.343 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.411 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.476 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.537 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.600 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.669 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.736 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.804 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.870 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.797.937 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.798.002 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.798.064 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.798.130 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.798.201 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.798.263 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.803.545 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.803.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.803.779 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.187 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.292 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.395 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.804.961 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.064 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.179 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.309 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.416 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.525 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.550 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.695 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.806 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.805.941 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.051 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.157 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.181 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.287 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.763 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.806.873 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.002 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.104 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.216 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.765 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.790 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.807.991 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.107 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.259 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.567 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffeda0ac360, registered count:5 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.601 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.624 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.643 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.667 [op_check.cc:71]619439 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.690 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.810 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.808.917 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.113 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.374 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.398 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.498 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.522 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.623 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.753 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.856 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.809.978 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.078 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.184 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.377 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.404 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.509 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.611 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.795 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.810.905 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.013 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.114 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.225 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.423 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.583 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.721 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.867 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.811.979 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.091 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.214 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.318 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.427 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.684 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.844 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.812.869 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffeda136e08, registered count:7 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.022 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.054 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffeda13f298, registered count:7 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.168 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.271 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.415 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.531 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.638 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.746 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.848 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.813.950 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.059 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.162 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.187 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.289 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.395 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.497 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.715 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.821 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.814.927 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.036 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.171 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.199 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.224 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.387 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.508 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.610 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.719 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.830 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.936 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.958 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.815.984 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.095 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.211 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.315 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.502 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.606 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.712 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.825 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.816.937 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.179 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.282 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.383 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.408 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.664 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.764 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.865 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.817.980 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.082 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.196 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.303 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.408 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.597 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.702 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.818.811 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.048 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.095 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.123 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.240 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.355 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.464 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.572 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.682 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.786 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.886 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.821.990 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.112 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.139 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.249 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.273 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.586 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.614 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.718 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.826 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.822.926 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.693 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.798 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.823.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.824.020 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.824.672 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.825.452 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.825.771 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.826.215 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.826.792 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.826.911 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.827.030 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.827.167 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.827.272 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.827.544 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.829.675 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.830.480 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.830.649 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.830.804 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.831.751 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.831.903 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.052 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.515 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.638 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.752 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.868 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.832.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.086 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.294 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.399 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.509 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.620 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.779 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.884 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.833.988 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.167 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.275 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.381 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.644 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.812 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.834.927 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.028 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.053 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.236 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.340 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.445 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.611 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.735 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.887 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.835.999 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.108 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.229 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.255 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.384 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.492 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.597 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.620 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.719 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.826 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.836.931 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.117 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.224 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.400 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.716 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.837.880 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.231 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.341 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.446 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.549 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.671 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.696 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.872 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.838.983 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.084 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.185 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.289 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.405 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.536 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.639 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.839.859 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.840.876 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.843.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.843.462 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.843.669 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.843.995 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.844.151 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.844.310 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.099 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.256 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.385 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.539 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.684 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.803 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.845.948 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.111 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.245 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.369 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.476 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.897 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.924 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.947 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.969 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.846.993 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.015 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.036 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.056 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.077 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.098 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.120 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.142 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.165 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.186 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.206 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.226 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.247 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.288 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.308 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.339 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.360 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.382 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.404 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.426 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.447 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.470 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.491 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.512 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.533 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.553 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.573 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.593 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.617 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.640 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.680 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.702 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.722 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.742 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.763 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.783 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.803 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.823 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.844 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.871 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.892 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.913 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.933 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.953 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.847.999 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.019 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.039 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.060 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.080 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.234 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.591 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.779 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.800 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.821 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.842 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.848.978 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.098 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.207 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.306 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.410 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.656 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.778 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.849.924 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.144 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.169 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.194 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.293 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.341 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.445 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.596 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.744 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.881 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.850.987 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.175 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.280 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.453 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.562 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.676 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.700 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.722 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.851.867 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.012 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.165 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.315 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.487 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.634 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.745 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.852.893 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.035 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.205 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.330 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.438 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.546 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.763 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.853.908 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.052 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.158 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.265 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.372 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.485 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.854.591 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.010 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.413 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.542 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.654 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.679 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.782 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.885 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.855.992 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.115 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.279 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.394 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.500 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.604 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.761 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.871 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.856.979 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.259 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.372 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.649 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.811 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.857.925 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.042 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.152 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.254 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.358 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.464 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.568 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.716 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.819 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.858.968 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.859.212 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.859.402 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.859.563 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.859.728 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.859.927 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.121 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.342 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.367 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.391 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.410 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.435 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.457 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.478 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.860.499 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.861.221 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.861.608 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.861.997 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.862.190 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.862.381 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.862.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.862.888 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.863.050 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.865.521 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.867.280 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.867.401 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.867.511 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.867.627 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.867.897 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.043 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.232 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.557 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.589 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.616 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.638 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.660 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.826 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.868.852 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.006 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.164 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.326 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.486 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.659 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.685 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.834 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.869.994 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.158 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.314 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.477 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.624 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.796 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.870.978 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.005 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.105 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.215 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.243 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.359 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.384 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.498 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.604 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.713 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.825 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.871.927 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.872.032 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.872.231 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.872.417 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.872.786 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.873.143 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.873.335 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.873.658 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.873.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.027 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.136 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.252 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.392 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.455 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.874.481 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.875.532 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.875.945 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.107 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.261 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.406 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.437 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.581 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.724 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.873 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.876.901 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.877.047 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.877.195 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.877.332 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.877.490 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.877.636 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.878.017 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.878.355 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.878.581 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.878.754 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.880.006 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.880.430 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.880.963 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.118 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.280 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.393 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.543 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.689 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.715 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.816 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.881.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.158 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.267 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.407 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.566 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.676 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.882.969 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.148 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.290 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.399 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.502 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.650 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.821 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.883.990 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.884.016 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.884.177 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.884.205 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.884.345 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.884.517 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.885.977 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.891.504 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.891.549 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.894.091 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.894.351 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.894.632 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.894.953 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.154 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.356 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.510 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.666 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.843 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.895.975 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.896.061 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.896.323 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.896.434 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.896.581 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.035 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.188 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.333 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.361 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.483 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.594 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.764 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.897.939 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.140 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.285 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.389 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.497 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.521 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.707 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.898.890 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.037 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.149 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.301 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.327 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.447 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.553 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.577 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.599 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.735 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.899.916 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.003 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.243 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.423 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.451 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.478 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.653 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.900.684 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.242 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.332 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.367 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.433 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.511 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.533 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.601 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.758 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.778 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.887 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.907 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.926 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.903.945 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.904.095 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.904.115 [tuning_tiling_registry.cc:30]619439 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.905.206 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffedaa7b0b0, registered count:7 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.905.245 [op_check.cc:24]619439 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffedaa7b310, registered count:5 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.905.627 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.906.125 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.907.918 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.908.078 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.908.171 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.908.530 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.908.960 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.938.669 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.938.788 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.938.959 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.939.046 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.939.191 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.939.329 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.939.973 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.055 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.319 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.403 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.547 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.770 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.940.947 [op_impl_registry.cc:122]619439 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619439,python3.9):2025-04-03-09:35:41.941.586 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.947.288 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619439,python3.9):2025-04-03-09:35:41.949.253 [op_impl_registry_holder_manager.cc:208]619439 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.024.895 [op_impl_space_registry.cc:109]619439 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.026.607 [op_impl_space_registry.cc:84]619439 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.026.639 [op_impl_space_registry.cc:85]619439 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.026.654 [op_impl_space_registry.cc:86]619439 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.026.669 [op_impl_space_registry.cc:87]619439 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.026.680 [op_impl_space_registry.cc:93]619439 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.027 [op_impl_space_registry.cc:273]619439 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.528 [plugin_manager.cc:564]619439 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.560 [plugin_manager.cc:134]619439 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.028.626 [file_utils.cc:47]619439 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.028.644 [plugin_manager.cc:145]619439 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619439,python3.9):2025-04-03-09:35:42.028.683 [plugin_manager.cc:567]619439 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.755 [mem_manager.cc:31]619439 Initialize:MemManager Initialize. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.791 [rts_caching_mem_allocator.cc:131]619439 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.809 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.824 [graph_mem_allocator.cc:17]619439 Initialize:MemoryAllocator::Initialize. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.841 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.857 [graph_mem_allocator.cc:17]619439 Initialize:MemoryAllocator::Initialize. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.870 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.885 [caching_allocator.cc:391]619439 FreeBlocks:Free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.898 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.917 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.932 [caching_allocator.cc:391]619439 FreeBlocks:Free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.941 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.956 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.971 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.028.985 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.001 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.016 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.029 [session_scope_mem_allocator.cc:73]619439 FreeAllMemory:Free all memory [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.045 [mem_manager.cc:102]619439 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.058 [session_scope_mem_allocator.cc:73]619439 FreeAllMemory:Free all memory [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.152 [dump_manager.cc:152]619439 Init:exception dump is not enabled [INFO] GE(619439,python3.9):2025-04-03-09:35:42.029.200 [ge_executor.cc:315]619439 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.222 [acl.cpp:321]619439 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.238 [dump.cpp:841]619439 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.397 [dump.cpp:759]619439 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.412 [dump.cpp:594]619439 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.426 [dump.cpp:224]619439 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.473 [dump.cpp:703]619439 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.493 [dump.cpp:771]619439 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.029.518 [dump.cpp:703]619439 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.596 [adx_datadump_server.cpp:77][tid:619439] start to do dump init [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.617 [adx_dump_record.cpp:210][tid:619439] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.634 [adx_dump_record.cpp:214][tid:619439] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.870 [adx_server_manager.cpp:149][tid:619550] server manager add component (10) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.871 [adx_dump_record.cpp:627][tid:619551] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.029.898 [adx_server_manager.cpp:165][tid:619550] server manager components init successfully [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.018 [adx_server_manager.cpp:203][tid:619552] Run Server(0) Process [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.098 [adx_server_manager.cpp:369][tid:619552] device up 0 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.130 [hdc_comm_opt.cpp:59][tid:619552] open device[0] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.251 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.270 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.287 [hdc_api.cpp:119][tid:619552] logDevId 0 create HDC server successfully [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.324 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.340 [adx_server_manager.cpp:369][tid:619552] device up 1 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.350 [hdc_comm_opt.cpp:59][tid:619552] open device[1] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.482 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.499 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.516 [hdc_api.cpp:119][tid:619552] logDevId 1 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.527 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.541 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.555 [adx_server_manager.cpp:369][tid:619552] device up 2 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.564 [hdc_comm_opt.cpp:59][tid:619552] open device[2] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.637 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.653 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.667 [hdc_api.cpp:119][tid:619552] logDevId 2 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.681 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.713 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.728 [adx_server_manager.cpp:369][tid:619552] device up 3 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.741 [hdc_comm_opt.cpp:59][tid:619552] open device[3] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.872 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.030.888 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.904 [hdc_api.cpp:119][tid:619552] logDevId 3 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.915 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.929 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.942 [adx_server_manager.cpp:369][tid:619552] device up 4 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.030.954 [hdc_comm_opt.cpp:59][tid:619552] open device[4] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.086 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.102 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.115 [hdc_api.cpp:119][tid:619552] logDevId 4 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.129 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.142 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.155 [adx_server_manager.cpp:369][tid:619552] device up 5 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.169 [hdc_comm_opt.cpp:59][tid:619552] open device[5] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.298 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.313 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.327 [hdc_api.cpp:119][tid:619552] logDevId 5 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.341 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.353 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.364 [adx_server_manager.cpp:369][tid:619552] device up 6 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.377 [hdc_comm_opt.cpp:59][tid:619552] open device[6] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.508 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.524 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.537 [hdc_api.cpp:119][tid:619552] logDevId 6 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.551 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.576 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.588 [adx_server_manager.cpp:369][tid:619552] device up 7 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.599 [hdc_comm_opt.cpp:59][tid:619552] open device[7] server[13] [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.730 [drv_log_user.c:623][ascend][curpid:619439,619552][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.031.746 [hdc_server.c:363][ascend][curpid:619439,619552][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.760 [hdc_api.cpp:119][tid:619552] logDevId 7 create HDC server successfully [WARNING] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.773 [adx_hdc_epoll.cpp:34][tid:619552] hdc epoll has been created. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.031.785 [adx_server_manager.cpp:111][tid:619552] create server info [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.032.978 [adx_server_manager.cpp:457][tid:619439] The server is initialized after waiting 3 times. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.033.000 [ge_executor.cc:1349]619439 SetDump:Start to set dump config [INFO] GE(619439,python3.9):2025-04-03-09:35:42.033.021 [dump_manager.cc:201]619439 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.033.061 [dump_manager.cc:261]619439 SetExceptionDump:Enable brief exception dump. [INFO] GE(619439,python3.9):2025-04-03-09:35:42.033.096 [ge_executor.cc:1355]619439 SetDump:Set dump config successfully [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.033.136 [dump_manager.cpp:146][tid:619439] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:42.034.104 [dump_manager.cpp:301][tid:619439] Register exception callback, registered: 1 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.123 [dump.cpp:825]619439 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.146 [acl.cpp:327]619439 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.161 [acl.cpp:329]619439 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.172 [acl_resource_manager.cpp:248]619439 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.268 [acl_resource_manager.cpp:275]619439 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.285 [acl_resource_manager.cpp:278]619439 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.297 [acl.cpp:335]619439 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.309 [acl.cpp:336]619439 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.373 [json_parser.cpp:260]619439 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.391 [acl.cpp:343]619439 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:42.034.407 [prof_atls_plugin.cpp:210] (tid:619439) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.423 [acl.cpp:351]619439 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.435 [profiling.cpp:141]619439 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.449 [profiling.cpp:135]619439 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.524 [profiling.cpp:167]619439 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619439.json], errorCode = 0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.541 [profiling.cpp:172]619439 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619439.json] is empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.552 [profiling.cpp:176]619439 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.564 [profiling.cpp:179]619439 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.034.576 [profiling.cpp:92]619439 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:42.034.972 [ai_drv_dev_api.cpp:316] >>> (tid:619439) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:42.035.089 [msprof_callback_impl.cpp:132] >>> (tid:619439) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.106 [profiling.cpp:122]619439 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.119 [profiling.cpp:186]619439 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.132 [hash_utils.cpp:108]619439 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619439.json [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.172 [hash_utils.cpp:128]619439 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619439.json, hash: 3917491750093904922 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.188 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.222 [acl.cpp:280]619439 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.237 [acl.cpp:385]619439 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.035.258 [runtime.cc:5394] 619439 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.035.276 [api_impl.cc:7407] 619439 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.035.301 [api_impl.cc:4211] 619439 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.035.320 [acl.cpp:420]619439 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.035.538 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.035.666 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.035.740 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.035.766 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:42.41097] [info] [619439] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:42.041.759 [prof_atls_plugin.cpp:210] (tid:619439) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.042.608 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.042.637 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.042.717 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 5, rank size = 8. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.042.736 [device.cpp:143]619439 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.764 [api_impl.cc:2999] 619439 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.781 [device.cc:306] 619439 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.800 [runtime.cc:3393] 619439 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.819 [runtime.cc:3560] 619439 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.836 [api_impl.cc:2987] 619439 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.853 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.873 [api_impl.cc:2992] 619439 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.042.896 [api_impl.cc:3025] 619439 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.042.913 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.042.927 [acl.cpp:121]619439 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.042.946 [device.cpp:154]619439 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.043.006 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.043.026 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is _npu_log, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.043.049 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.043.066 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.043.084 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.043.111 [queue_client.c:151][ascend][curpid:619439,619439][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.043.243 [queue_client.c:305][ascend][curpid:619439,619439][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=5; vfid=0; hostpid=619439; master_pid=619439; proc_type=0; devpid=31089). [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.047.207 [queue_client.c:362][ascend][curpid:619439,619439][drv][queuemng][QueueInit]QueueInit device. (devid=5; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.047.408 [npu_driver.cc:4114] 619439 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.047.431 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.047.448 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=_npu_log. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.047.760 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.047.779 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.047.907 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.047.940 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.047.969 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.047.971 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.047.993 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.047.997 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.013 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.026 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.039 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.068 [npu_driver.cc:4087] 619439 MemQueueInit: repeated init [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.085 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.102 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=ms_tensor_dump. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.048.512 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.048.532 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.048.709 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.048.735 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.048.744 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.753 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.048.764 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.767 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.795 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.809 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.821 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.847 [npu_driver.cc:4087] 619439 MemQueueInit: repeated init [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.863 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.048.877 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=ms_tensor_summary. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.049.047 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.065 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.160 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.191 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.049.194 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.209 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.216 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.223 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.238 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.253 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.265 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.291 [npu_driver.cc:4087] 619439 MemQueueInit: repeated init [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.306 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.318 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=ms_image_summary. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.049.474 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.491 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.587 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.049.613 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.624 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.633 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.646 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.652 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.660 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.666 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.679 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.705 [npu_driver.cc:4087] 619439 MemQueueInit: repeated init [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.721 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.049.733 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=ms_scalar_summary. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.049.883 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.049.901 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.003 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.050.022 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.040 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.050 [tensor_data_transfer.cpp:898]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.061 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.070 [api_impl.cc:6077] 619439 MemQueueInit: Start to init queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.074 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.085 [npu_driver.cc:4076] 619439 MemQueueInit: init mem queue, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.096 [npu_driver.cc:4026] 619439 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.121 [npu_driver.cc:4087] 619439 MemQueueInit: repeated init [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.137 [api_impl.cc:6040] 619439 MemQueueCreate: Start to create queue on devId 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.148 [npu_driver.cc:3931] 619439 MemQueueCreate: create mem queue, deviceId=5, name=ms_histogram_summary. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.050.302 [queue_interface.c:182][ascend][curpid:619439,619439][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.321 [tensor_data_transfer.cpp:940]619439 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 5, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.487 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.050.514 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.050.527 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.537 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.050.553 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.050.682 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.050.716 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.735 [plugin_manager.cc:25] [619439]hcom ops plugin init start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.773 [plugin_manager.cc:38] [619439]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.796 [plugin_manager.cc:42] [619439]hcom running normal mode. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.840 [hcom_plugin.cc:50] [619439]Initialize start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.856 [hcom_plugin.cc:52] [619439]initializeCount_ : 1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.894 [hcom_plugin.cc:60] [619439]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.923 [hcom_plugin.cc:63] [619439]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.940 [hcom_plugin.cc:329] [619439]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.050.958 [hcom_plugin.cc:281] [619439]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:42.050.982 [hcom_plugin.cc:217] [619439]option profiling mode is not found. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.001 [hcom_plugin.cc:164] [619439][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.018 [hcom_plugin.cc:180] [619439][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.035 [hcom_plugin.cc:244] [619439]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.162 [hcom_plugin.cc:82] [619439]hccl ops plugin init success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.181 [plugin_manager.cc:48] [619439]hcom ops plugin init end. ret[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.194 [hcom_plugin.cc:126] [619439]get hccl kernel info store start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.223 [hcom_plugin.cc:132] [619439]get hccl kernel info store finished. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.051.253 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.051.290 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.305 [hcom_executor.cc:29] [619439]Hcom Excutor Initialize start. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.367 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.386 [adapter_rts.cc:234] [619439][hrtGetDevice]deviceLogicId[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.439 [api_c_soc.cc:70] 619439 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.478 [hcom_common.cc:85] [619439][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.509 [hcom_common.cc:88] [619439][HcomGetCurHcomCtx] is no Used deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.537 [hcom.cc:486] [619439]Get Hcom Init Status: [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.554 [hcom_common.cc:85] [619439][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.051.566 [hcom_rpc.cc:1019] [619439]Get Hcom Init Status: [0] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:42.051.591 [hcom_base_funs.cc:36] [619439][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:42.051.612 [hcom_executor.cc:121] [619439][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.671 [api_impl.cc:1120] 619439 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.701 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.767 [stream.cc:2246] 619439 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.833 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.051.853 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.351 [npu_driver.cc:6125] 619439 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.405 [npu_driver.cc:6137] 619439 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=369, streamId=4, logicCq=369, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.437 [npu_driver.cc:6189] 619439 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=4, logicCq=369. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.458 [npu_driver.cc:5916] 619439 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c4089000, addr=dfffc4089000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.476 [stream.cc:653] 619439 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.052.551 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.571 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.052.918 [stream.cc:291] 619439 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.507 [stream.cc:688] 619439 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=5, streamResId=65535 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.053.529 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.053.557 [acl_resource_manager.cpp:816]619439 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.572 [api_impl.cc:1162] 619439 StreamCreate: Succ, flags=1536, stream_id=4, context=aaab18d025e0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.053.603 [stream.cc:73] [619439]Construct stream by stream type success, ptr[0xaaab231eb900] ctx[0xaaab18d025e0], stream id[4], cqId[4], logicCqid[369] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.643 [api_impl.cc:1120] 619439 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.669 [npu_driver.cc:404] 619439 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.701 [stream.cc:2246] 619439 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.739 [stream_sqcq_manage.cc:147] 619439 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.756 [npu_driver.cc:882] 619439 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.053.982 [npu_driver.cc:6125] 619439 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.036 [npu_driver.cc:6137] 619439 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=370, streamId=5, logicCq=370, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.058 [npu_driver.cc:6189] 619439 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=5, logicCq=370. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.076 [npu_driver.cc:5916] 619439 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c4068000, addr=dfffc4068000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.092 [stream.cc:653] 619439 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619439,python3.9):2025-04-03-09:35:42.054.140 [devmm_svm.c:1972][ascend][curpid:619439,619439][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.160 [npu_driver.cc:6080] 619439 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.054.465 [stream.cc:291] 619439 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.101 [stream.cc:688] 619439 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=5, streamResId=65535 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.055.121 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.055.136 [acl_resource_manager.cpp:816]619439 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.149 [api_impl.cc:1162] 619439 StreamCreate: Succ, flags=1536, stream_id=5, context=aaab18d025e0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.055.170 [stream.cc:73] [619439]Construct stream by stream type success, ptr[0xaaab23267960] ctx[0xaaab18d025e0], stream id[5], cqId[5], logicCqid[370] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:42.055.310 [sal.cc:619] [619565]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.055.348 [hcom_executor.cc:587] [619565]MessageProcessThreadLoop start queType:[0] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:42.055.351 [sal.cc:619] [619566]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.055.376 [hcom_executor.cc:587] [619566]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.378 [api_impl.cc:2999] 619565 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.392 [api_impl.cc:2999] 619567 SetDevice: device_id=5, deviceMode=0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:42.055.398 [hcom_executor.cc:34] [619439][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.408 [device.cc:306] 619565 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.409 [api_impl.cc:2999] 619566 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.437 [device.cc:306] 619566 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.437 [runtime.cc:3393] 619565 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.447 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.456 [runtime.cc:3393] 619566 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.458 [runtime.cc:3560] 619565 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.457 [device.cc:306] 619567 Init: device mode has been already set, mode=0. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.470 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.459 [api_impl.cc:2999] 619568 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.473 [runtime.cc:3560] 619566 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.477 [api_impl.cc:2987] 619565 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.485 [runtime.cc:3393] 619567 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.498 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.515 [device.cc:306] 619568 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.519 [api_impl.cc:2987] 619566 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.523 [runtime.cc:5239] 619565 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.527 [runtime.cc:3560] 619567 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.539 [runtime.cc:3393] 619568 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.544 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.545 [runtime.cc:5239] 619566 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.554 [api_impl.cc:2992] 619565 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.557 [api_impl.cc:2987] 619567 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.563 [runtime.cc:3560] 619568 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.576 [api_impl.cc:2992] 619566 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.581 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.607 [api_impl.cc:2987] 619568 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.585 [runtime.cc:5239] 619567 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:42.055.587 [awatchdog_core.c:175](tid:619565) create watchdog for id : 65543, tid : 619565, timeout : 300s [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:42.055.614 [awatchdog_core.c:175](tid:619566) create watchdog for id : 65543, tid : 619566, timeout : 300s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.623 [runtime.cc:5239] 619568 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.630 [api_impl.cc:2992] 619567 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.633 [api_impl.cc:3025] 619565 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.638 [api_impl.cc:3025] 619566 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.641 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1094.31 msec. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.648 [api_impl.cc:2992] 619568 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:42.055.652 [awatchdog_core.c:175](tid:619567) create watchdog for id : 65543, tid : 619567, timeout : 300s [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:42.055.675 [awatchdog_core.c:175](tid:619568) create watchdog for id : 65543, tid : 619568, timeout : 300s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.676 [api_impl.cc:3025] 619567 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.684 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.055.691 [api_impl.cc:3025] 619568 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [WARNING] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.713 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.768 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.015 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.820 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.863 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.055.888 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.096 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.056.017 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:42.056.045 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:42.056.134 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:42.056.183 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.056.236 [device.cpp:143]619569 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.267 [api_impl.cc:2999] 619569 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.283 [device.cc:306] 619569 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.298 [runtime.cc:3393] 619569 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.314 [runtime.cc:3560] 619569 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.328 [api_impl.cc:2987] 619569 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.343 [runtime.cc:5239] 619569 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.357 [api_impl.cc:2992] 619569 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:42.056.375 [awatchdog_core.c:175](tid:619569) create watchdog for id : 65543, tid : 619569, timeout : 300s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.056.390 [api_impl.cc:3025] 619569 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.056.403 [acl.cpp:115]619569 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.056.416 [acl.cpp:121]619569 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.056.430 [device.cpp:154]619569 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:42.056.468 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.223 msec. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:42.056.755 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 1 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.244.172 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.244.172 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.244.196 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.244.204 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.214 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.231 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.237 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.247 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.244.257 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.244.259 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.244.277 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.244.277 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.297 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.244.297 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.412.728 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:42.412.757 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:42.412.778 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:42.412.795 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.412.818 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.412.895 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:42.556.939 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 398/400, sleep 2 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.756.174 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.756.197 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.756.203 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.213 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.756.235 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.240 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.251 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.756.259 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.283 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.756.287 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.756.301 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.304 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.756.319 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.756.333 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.169 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.169 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.169 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.169 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.171 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:42.884.170 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.266 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.283 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.298 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.300 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.313 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.320 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.326 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.331 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.335 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.339 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.346 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.350 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.366 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.376 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.382 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.388 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.392 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.398 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.404 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.405 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.410 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.418 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.425 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.429 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.431 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.435 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.439 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.450 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.457 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.460 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.467 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.476 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:42.884.479 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.489 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.495 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:42.884.508 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:43.057.116 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 397/400, sleep 1 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.268.178 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.268.182 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.268.199 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.268.204 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.211 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.221 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.228 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.234 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.268.242 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.268.254 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.268.249 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.268 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.268.274 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.268.287 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:43.557.285 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 396/400, sleep 2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.628.712 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:43.628.738 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:43.628.753 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:43.628.769 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.628.786 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.628.850 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.171 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.172 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.173 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.179 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.179 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.716.217 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.281 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.301 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.309 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.313 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.323 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.328 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.336 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.337 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.341 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.350 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.356 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.360 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.366 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.368 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.371 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.379 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.387 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.391 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.400 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.404 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.404 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.437 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.443 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.446 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.444 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.460 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.464 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.484 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.485 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.492 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.498 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.515 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.521 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.716.532 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.554 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.716.574 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.780.177 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.780.189 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.780.204 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:43.780.214 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.222 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.231 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.236 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.247 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.780.253 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.780.283 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.780.288 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:43.780.300 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.305 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:43.780.317 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:44.057.564 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 395/400, sleep 1 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.292.183 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.292.184 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.292.236 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.292.243 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.255 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.264 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.274 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.281 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.292.295 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.292.304 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.292.318 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.292.320 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.341 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.292.341 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.544.197 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.544.382 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.544.407 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.544.447 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.544.459 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.544.474 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.544.489 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.548.184 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.548.181 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.548.184 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.548.263 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.548.263 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.315 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.337 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.357 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.359 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.375 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.377 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.385 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.392 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.401 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.406 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.412 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.417 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.418 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.424 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.432 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.438 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.445 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.450 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.474 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.481 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.491 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.495 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.502 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.510 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.517 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.522 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.548.526 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.537 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.542 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.548.559 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [WARNING] DEVICE(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:44.557.844 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 394/400, sleep 2 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.804.178 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.804.227 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.804.223 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.247 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:44.804.265 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.273 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.279 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.804.292 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.294 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.804.314 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.804.343 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.353 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:44.804.361 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.804.377 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.840.749 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:44.840.790 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:44.840.810 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:44.840.827 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.840.858 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:44.840.996 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:45.058.146 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:45.058.204 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 3001.7 msec. [WARNING] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:45.058.227 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:45.058.256 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.058.495 [device.cpp:143]619756 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.534 [api_impl.cc:2999] 619756 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.553 [device.cc:306] 619756 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.571 [runtime.cc:3393] 619756 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.589 [runtime.cc:3560] 619756 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.604 [api_impl.cc:2987] 619756 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.622 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.643 [api_impl.cc:2992] 619756 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:45.058.667 [awatchdog_core.c:175](tid:619756) create watchdog for id : 65543, tid : 619756, timeout : 300s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.058.695 [api_impl.cc:3025] 619756 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.058.709 [acl.cpp:115]619756 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.058.725 [acl.cpp:121]619756 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.058.741 [device.cpp:154]619756 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.308.351 [device.cpp:231]619756 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.385 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.573 [api_c_soc.cc:70] 619756 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.308.639 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.661 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.687 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.699 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.713 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.308.727 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=0. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.308.954 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=0.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.308.995 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.015 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.031 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.043 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.057 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.069 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=1. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.309.358 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=1.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.309.396 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.414 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.432 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.445 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.458 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.309.469 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=2. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.310.103 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=2.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.310.140 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.158 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.174 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.190 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.203 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.215 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=3. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.310.831 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=3.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.310.868 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.885 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.899 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.912 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.925 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.310.937 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=4. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.311.964 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=4.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.312.001 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.312.018 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.312.032 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.312.049 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.312.063 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.312.076 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=6. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.313.016 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=6.) [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.313.061 [memory.cpp:559]619756 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.083 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.097 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.111 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.122 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.149 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=7. [INFO] DRV(619439,python3.9):2025-04-03-09:35:45.313.956 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=7.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.313.994 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.066 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.093 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.147 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.182 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.197 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.215 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.232 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.249 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.274 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.294 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.309 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.332 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.349 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.365 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.380 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.397 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.440 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.473 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.489 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.505 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.519 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.534 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.551 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.577 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.623 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.653 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.673 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.690 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.705 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.725 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.744 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.761 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.778 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.808 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.826 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.844 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.860 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.887 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.906 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.923 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.960 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.314.984 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.002 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.018 [runtime.cc:1664] 619756 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.036 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.062 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.079 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.100 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.141 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.164 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.178 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.199 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.213 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.234 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.256 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.271 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.282 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.297 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.312 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.346 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.361 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.376 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.390 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.406 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.437 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.451 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.463 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.475 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.489 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.518 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.535 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.548 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.563 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.578 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.608 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.624 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.638 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.655 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.678 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.702 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.716 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.315.734 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.316.168 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.316.192 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.316.200 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.316.219 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.229 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.238 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.245 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.256 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.316.266 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.316.276 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.316.281 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.316.292 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.302 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.309 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.527 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.584 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.638 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.654 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.667 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.681 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.705 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.756 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.771 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.781 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.796 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.818 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.853 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.866 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.878 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.894 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.922 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.964 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.978 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.316.992 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.008 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.025 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.067 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.080 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.092 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.106 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.129 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.166 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.182 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.194 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.212 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.228 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.272 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.286 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.305 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.320 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.346 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.385 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.399 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.413 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.427 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.441 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.481 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.495 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.511 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.525 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.549 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.592 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.606 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.620 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.638 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.654 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.691 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.706 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.718 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.733 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.759 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.798 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.813 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.826 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.848 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.864 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.902 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.917 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.932 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.949 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.317.972 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.014 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.031 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.046 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.059 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.074 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.114 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.127 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.142 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.156 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.177 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.215 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.231 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.248 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.263 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.279 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.323 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.338 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.352 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.368 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.387 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.436 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.453 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.466 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.483 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.499 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.536 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.553 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.567 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.582 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.599 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.644 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.659 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.673 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.688 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.711 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.749 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.764 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.779 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.794 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.811 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.849 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.866 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.880 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.318.920 [api_impl.cc:1990] 619756 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.319.242 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.319.273 [api_impl.cc:2664] 619756 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.550 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619419, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.586 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619423, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.606 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619427, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.622 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619431, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.636 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619435, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.657 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619443, num=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.675 [npu_driver.cc:3571] 619756 SetIpcMemPid: set ipc mem pid success, name=000973af000000000000031c0500ffd40005, pid[0]=619447, num=1 [2025-04-03 09:35:42.41786] [debug] [619439] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:42.41799] [debug] [619439] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.58872] [warn] [619756] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.58898] [debug] [619756] [lcal_sock_exchange.cpp:140] curRank: 5 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.58905] [debug] [619756] [lcal_sock_exchange.cpp:272] Client side 5 begin to connect [2025-04-03 09:35:45.59086] [debug] [619756] [lcal_sock_exchange.cpp:93] rank:5 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308312] [debug] [619756] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 5 rank :5 rankSize :8 [2025-04-03 09:35:45.308514] [debug] [619756] [lcal_comm.cpp:406] rank 5 devId: 5, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308529] [info] [619756] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308537] [info] [619756] [lcal_comm.cpp:242] rank 5/8 running devId:5 [2025-04-03 09:35:45.308594] [debug] [619756] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308628] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308984] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309386] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310131] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310859] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311992] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313047] [debug] [619756] [lcal_comm.cpp:340] 5 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313974] [debug] [619756] [lcal_comm.cpp:372] EnablePeerAccess succeed5 [2025-04-03 09:35:45.318898] [debug] [619756] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 5 [2025-04-03 09:35:45.318911] [debug] [619756] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.319261] [debug] [619756] [lcal_comm.cpp:455] peerMem[rank5], allocate finished. [2025-04-03 09:35:45.340424] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340460] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340467] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340474] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340480] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340487] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340493] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340498] [debug] [619756] [lcal_comm.cpp:472] rank : 5, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340506] [debug] [619756] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340691] [debug] [619756] [lcal_comm.cpp:557] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340806] [debug] [619756] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340816] [debug] [619756] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340823] [debug] [619756] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340830] [debug] [619756] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340851] [debug] [619756] [lcal_comm.cpp:515] rank 4 mem[INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.340.909 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.341.982 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.342.001 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.342.807 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.342.823 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.343.274 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.343.291 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.343.692 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.343.709 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.357.947 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.357.972 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.358.541 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.358.559 [api_impl.cc:4286] 619756 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.257 [npu_driver.cc:3195] 619756 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.296 [api_impl.cc:1990] 619756 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.320 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.359.340 [memory.cpp:421]619756 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:45.359.523 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.359.565 [device.cpp:143]619756 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.588 [api_impl.cc:2999] 619756 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.604 [device.cc:306] 619756 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.617 [runtime.cc:3393] 619756 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.634 [runtime.cc:3560] 619756 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.651 [api_impl.cc:2987] 619756 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.668 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.685 [api_impl.cc:2992] 619756 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.702 [api_impl.cc:3025] 619756 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.359.716 [acl.cpp:115]619756 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.359.733 [acl.cpp:121]619756 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.359.747 [device.cpp:154]619756 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [WARNING] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:45.359.781 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.359.816 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.838 [adapter_rts.cc:198] [619756][hrtGetDeviceRefresh]deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.856 [externalinput.cc:2065] [619756]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.877 [externalinput.cc:490] [619756]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.894 [externalinput.cc:429] [619756]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.907 [externalinput.cc:543] [619756]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.925 [externalinput.cc:622] [619756]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.941 [externalinput.cc:714] [619756]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.957 [externalinput.cc:821] [619756]HCCL_IF_IP is not set [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.971 [externalinput.cc:877] [619756]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.985 [externalinput.cc:844] [619756]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.359.997 [externalinput.cc:728] [619756]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.025 [externalinput.cc:758] [619756]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.038 [externalinput.cc:806] [619756]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.051 [externalinput.cc:1508] [619756]HCCL_ALGO is not set [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.068 [externalinput.cc:1671] [619756][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.083 [externalinput.cc:462] [619756]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.097 [externalinput.cc:1519] [619756]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.109 [externalinput.cc:1554] [619756]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.360.278 [api_c_soc.cc:70] 619756 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.303 [externalinput.cc:1599] [619756]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.326 [externalinput.cc:1633] [619756]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.345 [externalinput.cc:1778] [619756]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.360 [externalinput.cc:1807] [619756]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.374 [externalinput.cc:515] [619756]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.385 [externalinput.cc:1841] [619756]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.399 [externalinput.cc:1929] [619756]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.411 [externalinput.cc:1949] [619756]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.423 [externalinput.cc:1969] [619756]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.445 [externalinput.cc:2005] [619756]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.461 [externalinput.cc:2036] [619756]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.477 [externalinput.cc:2209] [619756]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.492 [externalinput.cc:2178] [619756][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.506 [externalinput.cc:889] [619756]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.519 [externalinput.cc:914] [619756]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.533 [externalinput.cc:2240] [619756]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.546 [externalinput.cc:2308] [619756]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.559 [externalinput.cc:2339] [619756]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.573 [externalinput.cc:2296] [619756]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.589 [externalinput.cc:407] [619756]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:45.360.604 [externalinput.cc:672] [619756][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:45.360.645 [env_config.cc:127] [619756]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:45.360.665 [env_config.cc:131] [619756]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.682 [env_config.cc:302] [619756]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.696 [env_config.cc:302] [619756]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.707 [op_base.cc:1091] [619756]Entry-HcclCommInitRootInfoInner:ranks[8], rank[5], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.734 [op_base.cc:128] [619756][HcclGetDeviceId] deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.748 [op_base.cc:144] [619756][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.360.773 [rank_consistentcy_checker.cc:32] [619756][GetInstance] get deviceLogicId[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.360.808 [api_impl.cc:3100] 619756 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.360.826 [api_impl.cc:3111] 619756 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.360.841 [api_impl.cc:3130] 619756 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.361.055 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.361.078 [network_manager.cc:169] [619756]NetworkManager: devicePhyId[5], devicePhyId_[5] deviceLogicId_[5], hasBackup[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.103 [ra_host.c:1594]tid:619756,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.361.123 [adapter_hccp.cc:1450] [619756]set host socket whitelist status[0] success. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.155 [ra_host.c:205]tid:619756,ra_init(205) : Input parameters: phy_id[5], nic_position:[0] hdc_type:[6] [INFO] ROCE(619439,python3.9):2025-04-03-09:35:45.361.209 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.225 [ra_peer.c:776]tid:619756,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[5] start [INFO] ROCE(619439,python3.9):2025-04-03-09:35:45.361.275 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619439,python3.9):2025-04-03-09:35:45.361.334 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.365 [rs.c:327]tid:619756,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619439,python3.9):2025-04-03-09:35:45.361.502 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619439,python3.9):2025-04-03-09:35:45.361.524 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619439,python3.9):2025-04-03-09:35:45.361.539 [rs_ssl.c:1513]tid:619756,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.552 [rs_ssl.c:1529]tid:619756,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.582 [rs_epoll.c:692]tid:619756,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.687 [rs_epoll.c:487]tid:619763,rs_epoll_handle(487) : thread begin! thread_id:281468592910240, pid:619439, ppid:619411 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.720 [rs_epoll.c:713]tid:619756,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.737 [rs_epoll.c:623]tid:619764,rs_connect_handle(623) : thread begin! thread_id:281468584456096, pid:619439, ppid:619411 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.735 [rs.c:366]tid:619756,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.167000] ms [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.762 [rs_epoll.c:415]tid:619763,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.780 [rs_epoll.c:511]tid:619763,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.361.781 [rs_epoll.c:641]tid:619764,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.167 [rs.c:471]tid:619756,rs_init(471) : rs init success, chip_id[5] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.192 [ra_peer.c:813]tid:619756,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[5] succ [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.370.208 [adapter_hccp.cc:583] [619756]init ra success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.370.225 [network_manager.cc:222] [619756]NetworkManager nicDeploy[0] deviceLogicId_[5] devicePhyId_[5] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.256 [ra_host.c:1764]tid:619756,ra_get_ifnum(1764) : Input parameters: phy_id[5], nic_position:[0], is_all:[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.274 [ra_peer.c:859]tid:619756,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[5] start [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.295 [ra_peer.c:867]tid:619756,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[5] succ [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.370.308 [adapter_hccp.cc:1378] [619756][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.324 [ra_host.c:1814]tid:619756,ra_get_ifaddrs(1814) : Input parameters: phy_id[5], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.339 [ra_peer.c:874]tid:619756,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[5] start [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.352 [rs_socket.c:2743]tid:619756,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.367 [rs_socket.c:2743]tid:619756,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.382 [rs_socket.c:2743]tid:619756,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.394 [rs_socket.c:2749]tid:619756,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.406 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.419 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.432 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.446 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.456 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.466 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.477 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.488 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.515 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.530 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.541 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.554 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.568 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.582 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.595 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.609 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.620 [rs_socket.c:2749]tid:619756,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.633 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.647 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.662 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.673 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.683 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.694 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.705 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.720 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.733 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.746 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.755 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.767 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.778 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.790 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.803 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.813 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.824 [rs_socket.c:2749]tid:619756,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.835 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.846 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.857 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.875 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.894 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.906 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.917 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.930 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.946 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.960 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.976 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.370.988 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.002 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.016 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.028 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.042 [rs_socket.c:2751]tid:619756,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.052 [ra_peer.c:882]tid:619756,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[5] succ [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.076 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.092 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.106 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.122 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.136 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.149 [adapter_hccp.cc:1407] [619756][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.170 [sal.cc:365] [619756]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.183 [sal.cc:432] [619756]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.205 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:45.371.229 [ra_host.c:340]tid:619756,ra_socket_init_v1(340) : socket init:mode=0 phy_id=5 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.243 [adapter_hccp.cc:818] [619756]socket init v1 success, socketHandle[0xfffe7c0faba0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.253 [network_manager.cc:1218] [619756]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.263 [network_manager.cc:1257] [619756]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe7c0faba0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.322 [adapter_hal.cc:298] [619756]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.345 [sal.cc:593] [619756]IsHostUseDevNic[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.368 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.382 [network_manager.cc:169] [619756]NetworkManager: devicePhyId[4294967295], devicePhyId_[5] deviceLogicId_[5], hasBackup[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.427 [tsd_client.cpp:189][TsdCapabilityGet][tid:619756] TsdCapabilityGet Begin. [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.441 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.455 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.469 [process_mode_manager.cpp:1042][CapabilityGet][tid:619756] [ProcessModeManager] enter into CapabilityGet process deviceId[5] type[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.484 [adapter_tdt.cc:66] [619756]Get TsdCapability success. deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.497 [network_manager.cc:175] [619756][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:45.371.518 [network_manager.cc:99] [619756][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.535 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.547 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.562 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619756] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.574 [process_mode_manager.cpp:386][InitTsdClient][tid:619756] [TsdClient] tsd client has been inited before [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.588 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619756] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619439,python3.9):2025-04-03-09:35:45.371.605 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619756] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.619 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619756] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.634 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619756] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.648 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619756] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.668 [version_verify.cpp:114][SpecialFeatureCheck][tid:619756] VersionVerify: previous type[48], supported [INFO] TDT(619439,python3.9):2025-04-03-09:35:45.371.706 [hdc_client.cpp:154][TsdRecvData][tid:619756] recv timeout is:150000 ms [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.173 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.173 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.174 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.188 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.255 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.380.258 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.313 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.334 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.344 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.351 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.360 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.365 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.345 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.377 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.378 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.384 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.415 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.423 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.438 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.441 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.442 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.450 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.457 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.469 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.467 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.491 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.497 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.504 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.567 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.529 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.556 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.592 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.600 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.556 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.581 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.606 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.380.617 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.655 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.631 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.637 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.671 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.380.695 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.828.168 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.828.171 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.828.199 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:45.828.209 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.217 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.224 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.232 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.239 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.828.270 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.828.278 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.828.285 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:45.828.299 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.306 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:45.828.316 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.060.663 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:46.060.690 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:46.060.703 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:46.060.718 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.176 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.176 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.177 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.183 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.184 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.212.183 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.282 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.300 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.316 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.319 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.332 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.338 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.345 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.350 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.367 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.373 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.380 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.384 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.386 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.396 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.405 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.417 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.419 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.422 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.433 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.443 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.452 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.443 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.460 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.467 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.484 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.510 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.516 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.491 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.501 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.523 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.532 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.538 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.564 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.212.636 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.650 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.212.680 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.340.171 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.340.171 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.340.198 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.340.207 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.215 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.223 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.233 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.241 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.340.248 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.340.256 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.340.261 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.340.272 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.280 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.340.288 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.852.176 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.852.185 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.852.206 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:46.852.218 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.223 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.239 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.243 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.256 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.852.280 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.852.286 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.852.296 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:46.852.302 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.314 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:46.852.322 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.170 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.171 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.174 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.182 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.177 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.044.183 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.277 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.296 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.302 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.314 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.320 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.331 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.332 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.337 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.344 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.352 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.356 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.360 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.364 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.388 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.392 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.400 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.401 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.416 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.440 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.422 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.422 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.433 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.448 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.455 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.500 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.515 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.465 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.485 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.491 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.533 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.540 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.591 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.603 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.044.580 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.623 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.044.663 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.364.169 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.364.179 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.364.197 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.364.204 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.214 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.219 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.231 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.235 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.364.246 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.364.253 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.364.261 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.364.265 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.278 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.364.282 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.731.832 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619756] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.731.836 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.731.861 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619756] [TsdClient] DeviceMsgProc recvMsg realDeviceId[5] msgType[49] localDevId[5] rspCode[0]heterogeneousSubPid[31120], tsdSupportLevel_[63] [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.731.881 [hdc_client.cpp:172][TsdRecvData][tid:619756] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.731.896 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619756] OpenSubProc success stype:0, pid:31120 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.731.910 [adapter_tdt.cc:39] [619756]Open TsdProcess success. deviceLogicId[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.731.919 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.731.925 [network_manager.cc:118] [619756][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31120], devicePhyId_[5], deviceLogicId_[5], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.731.943 [network_manager.cc:180] [619756][NetworkManager][Init]open tsd by process success, devicePhyId[5], deviceLogicId_[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.731.985 [ra_host.c:205]tid:619756,ra_init(205) : Input parameters: phy_id[5], nic_position:[1] hdc_type:[18] [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.732.002 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619439,python3.9):2025-04-03-09:35:47.732.040 [devdrv_hccl.c:103][ascend][curpid:619439,619756][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619439) [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.732.059 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.732.077 [ra_hdc.c:580]tid:619756,ra_hdc_init(580) : hdc init start! logic id is 5, phy id is 5, hdc_type is 18 [INFO] DRV(619439,python3.9):2025-04-03-09:35:47.732.488 [hdc_core.c:1982][ascend][curpid:619439,619756][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4268; pid=619439) [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.732.792 [ra_hdc.c:616]tid:619756,ra_hdc_init(616) : hdc init OK! phy_id[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.741.355 [adapter_hccp.cc:583] [619756]init ra success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.741.373 [network_manager.cc:222] [619756]NetworkManager nicDeploy[1] deviceLogicId_[5] devicePhyId_[5] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.741.400 [topoinfo_detect.cc:534] [619756]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.741.419 [ra_host.c:1764]tid:619756,ra_get_ifnum(1764) : Input parameters: phy_id[5], nic_position:[1], is_all:[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.741.851 [adapter_hccp.cc:1538] [619756][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.741.868 [ra_host.c:1814]tid:619756,ra_get_ifaddrs(1814) : Input parameters: phy_id[5], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.237 [adapter_hccp.cc:1568] [619756][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth5] ip[29.105.72.114%eth5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.256 [adapter_hccp.cc:1568] [619756][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth5] ip[fe80::2287:ecff:fe02:4618%eth5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.274 [topoinfo_detect.cc:577] [619756]select AF_INET family as device socket family. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.299 [topoinfo_detect.cc:756] [619756]no backup device ip: use 0 as device ip. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.314 [topoinfo_detect.cc:294] [619756][HCCL_TRACE]SetupAgent rankNum[8], rank[5], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[5], phydevId[5], deviceIp[29.105.72.114] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.346 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.362 [hccl_socket.cc:516] [619756][Get][NicHandleInfo]phyId[5], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.379 [hccl_socket.cc:217] [619756][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281468468177824], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.742.396 [adapter_hccp.cc:1050] [619756]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.411 [ra_host.c:759]tid:619756,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[5], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.441 [rs_socket.c:1810]tid:619756,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.459 [rs_socket.c:1819]tid:619756,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619439,python3.9):2025-04-03-09:35:47.742.484 [rs_socket.c:132]tid:619756,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.502 [rs_socket.c:1880]tid:619756,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.782 [rs_socket.c:1577]tid:619764,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:35355, fd:53 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.810 [rs_drv_socket.c:59]tid:619764,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:53 begin [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.900 [rs_drv_socket.c:98]tid:619764,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:35355, server addr 192.168.0.44:60008, fd:53 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.922 [rs_socket.c:1427]tid:619764,rs_conn_cost_time(1427) : socket [53] connect success! cost [0.420000] ms [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.742.976 [rs_socket.c:1392]tid:619764,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.743.584 [topoinfo_exchange_agent.cc:212] [619756]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.743.642 [topoinfo_exchange_agent.cc:236] [619756]local rank[5] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.743.660 [topoinfo_exchange_agent.cc:43] [619756]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.743.675 [topoinfo_exchange_agent.cc:345] [619756]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.743.768 [topoinfo_exchange_agent.cc:77] [619756]topo exchange client send rank basic info success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.800.669 [api_impl.cc:2096] 619756 HostMalloc: size=2485 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.800.710 [api_error.cc:1260] 619756 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.800.978 [api_error.cc:1278] 619756 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.800.997 [topoinfo_exchange_agent.cc:80] [619756]topo exchange client get rank basic info success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.042 [topoinfo_detect.cc:303] [619756]topo detect completed. myrank[5], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.081 [config.cc:198] [619756]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.104 [config.cc:210] [619756]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.120 [config.cc:210] [619756]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.133 [config.cc:210] [619756]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.146 [config.cc:210] [619756]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.170 [config.cc:210] [619756]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.182 [config.cc:210] [619756]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.193 [config.cc:210] [619756]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.208 [config.cc:210] [619756]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.801.220 [adapter_hccp.cc:1067] [619756]ra socket batch close fdhandle[0xfffe7c054bf0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.801.234 [ra_host.c:796]tid:619756,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[5], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.801.252 [rs_socket.c:1937]tid:619756,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:53, state:9 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.848.205 [adapter_hccp.cc:1099] [619756]ra socket batch close success,take time [46972]us [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:47.848.241 [hccl_socket.cc:232] [619756][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.848.265 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.284 [ra_host.c:404]tid:619756,ra_socket_deinit(404) : Input parameters: phy_id[5] family[2] local_ip[192.168.0.44] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.304 [rs.c:1629]tid:619756,rs_socket_deinit(1629) : rs socket deinit start, phy_id:5 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.318 [rs.c:1641]tid:619756,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.333 [rs.c:1653]tid:619756,rs_socket_deinit(1653) : socket deinit success, phy_id:5, local_ip:192.168.0.44 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.848.346 [network_manager.cc:1233] [619756]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.848.358 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.370 [ra_host.c:248]tid:619756,ra_deinit(248) : Input parameters: phy_id[5], nic_position:[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.385 [ra_peer.c:821]tid:619756,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[5] start [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.406 [rs.c:1783]tid:619756,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619439,python3.9):2025-04-03-09:35:47.848.409 [rs_epoll.c:523]tid:619763,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:51 [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.848.435 [rs_epoll.c:544]tid:619763,rs_epoll_handle(544) : QUIT thread_id:281468592910240, pid:619439, read num:8 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.176 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.177 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.179 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.181 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.184 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.183 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.183 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.247 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.263 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:47.876.271 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.305 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.289 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.314 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.321 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.324 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.338 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.346 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.360 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.367 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.346 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.352 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.356 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.373 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.386 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.387 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.393 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.402 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.411 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.417 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.430 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.429 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.436 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.458 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.459 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.473 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.486 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.493 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.502 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.510 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.549 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.517 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.526 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.532 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.537 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.556 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.570 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.579 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.587 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.593 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:47.876.611 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.620 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.627 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.632 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.641 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.659 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:47.876.695 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.953.430 [rs_epoll.c:679]tid:619764,rs_connect_handle(679) : QUIT thread_id:281468584456096, pid:619439 [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.968.761 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.968.788 [rs.c:1810]tid:619756,rs_deinit(1810) : rs_deinit chip_id[5] ok [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.968.804 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.968.818 [ra_peer.c:852]tid:619756,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[5] succ [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.968.832 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.968.859 [network_manager.cc:409] [619756]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.968.877 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.968.893 [ra_host.c:248]tid:619756,ra_deinit(248) : Input parameters: phy_id[5], nic_position:[1] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.968.908 [ra_hdc.c:651]tid:619756,ra_hdc_deinit(651) : hdc deinit start! phy_id[5] [INFO] DRV(619439,python3.9):2025-04-03-09:35:47.969.102 [hdc_client.c:417][ascend][curpid:619439,619756][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4268) [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.969.168 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:47.969.185 [ra_hdc.c:685]tid:619756,ra_hdc_deinit(685) : hdc deinit OK! phy_id[5] [INFO] ROCE(619439,python3.9):2025-04-03-09:35:47.969.199 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.225 [tsd_client.cpp:189][TsdCapabilityGet][tid:619756] TsdCapabilityGet Begin. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.241 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.259 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.272 [process_mode_manager.cpp:1042][CapabilityGet][tid:619756] [ProcessModeManager] enter into CapabilityGet process deviceId[5] type[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:47.969.288 [adapter_tdt.cc:66] [619756]Get TsdCapability success. deviceLogicId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.302 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.318 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.332 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619756] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.349 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619756] add close subproc:31120, proctype:0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.365 [version_verify.cpp:114][SpecialFeatureCheck][tid:619756] VersionVerify: previous type[60], supported [INFO] TDT(619439,python3.9):2025-04-03-09:35:47.969.407 [hdc_client.cpp:154][TsdRecvData][tid:619756] recv timeout is:150000 ms [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.201.458 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619756] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.201.484 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619756] [TsdClient] DeviceMsgProc recvMsg realDeviceId[5] msgType[61] localDevId[5] rspCode[0]heterogeneousSubPid[31120], tsdSupportLevel_[63] [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.201.503 [hdc_client.cpp:172][TsdRecvData][tid:619756] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.201.519 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619756] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.201.534 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619756] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.547 [adapter_tdt.cc:52] [619756]Open TsdProcess success. deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.561 [network_manager.cc:405] [619756][DeInit]finish CloseHccpProcess, phy_id[5], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.577 [network_manager.cc:409] [619756]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.588 [topoinfo_detect.cc:555] [619756]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.603 [topoinfo_detect.cc:321] [619756]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.614 [topoinfo_detect.cc:219] [619756]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.630 [rank_consistentcy_checker.cc:32] [619756][GetInstance] get deviceLogicId[5] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:48.201.644 [rank_consistentcy_checker.cc:144] [619756][Record][CannVersion] version information str is empty. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.655 [op_base.cc:2231] [619756]rank table is null, rankTableCrc is 0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.689 [op_base.cc:2219] [619756]params.id.internal [hccl-619439-0-1743644148-hccl_world_group] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.705 [op_base.cc:993] [619756]rootInfo[hccl-619439-0-1743644148-hccl_world_group], params.logiceDevice[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.720 [hccl_comm.cc:98] [619756]hcclComm init workmode [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.201.732 [hccl_comm.cc:954] [619756]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.202.577 [hccl_comm.cc:1365] [619756]RegistTaskAbortHandler begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.202.597 [task_abort_handler.cc:126] [619756]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.202.614 [api_impl.cc:4192] 619756 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.202.631 [runtime.cc:4698] 619756 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.202.675 [adapter_rts.cc:42] [619756]open so: libruntime.so [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.202.938 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.202.975 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.202.995 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.029 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.047 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.069 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.082 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.101 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.117 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.138 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.156 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.179 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.194 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.215 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.230 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.251 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.263 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.281 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.294 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.311 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.325 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.342 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.356 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.375 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.389 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.405 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.420 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.447 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.460 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.482 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.496 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.514 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.528 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.547 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.559 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.574 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.591 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.609 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.623 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.641 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.656 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.676 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.693 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.712 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.724 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.740 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.753 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.773 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.786 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.805 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.817 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.839 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.862 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.881 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.203.897 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.918 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.962 [hccl_communicator_attrs.cc:407] [619756]module[0] contains [8]devices [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.977 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.203.989 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.001 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.017 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.028 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.041 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.054 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.066 [hccl_communicator_attrs.cc:411] [619756]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.079 [hccl_communicator_attrs.cc:425] [619756]different module contains different numbers of cards:[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.090 [hccl_communicator_attrs.cc:426] [619756]different module contains different type of cards:[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.140 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.175 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.201 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.221 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.234 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.255 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.271 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.289 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.303 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.322 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.350 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.369 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.386 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.408 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.421 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.444 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.458 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.477 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.490 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.507 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.521 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.539 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.551 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.570 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.585 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.603 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.616 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.633 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.649 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.666 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.680 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.699 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.713 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.731 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.750 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.768 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.784 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.803 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.817 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.835 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.853 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.874 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.887 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.905 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.920 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.939 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.953 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.204.974 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.204.989 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.010 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.022 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.044 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.059 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.076 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.089 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.108 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.119 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.136 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.150 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.181 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.193 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.215 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.230 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.246 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.262 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.280 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.293 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.312 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.326 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.344 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.359 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.377 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.392 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.409 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.424 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.443 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.456 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.474 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.492 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.511 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.523 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.544 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.557 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.575 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.593 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.613 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.626 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.643 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.657 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.674 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.686 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.701 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.717 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.739 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.755 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.775 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.789 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.806 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.821 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.841 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.855 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.873 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.886 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.907 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.919 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.936 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.948 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.205.968 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.205.981 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.008 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.023 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.045 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.063 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.087 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.098 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.115 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.126 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.150 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.164 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.182 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.196 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.216 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.229 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.247 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.259 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.278 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.290 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.307 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.317 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.336 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.350 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.370 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.381 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.404 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.420 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.436 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.447 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.467 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.484 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.500 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.515 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.531 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.546 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.565 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.578 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.595 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.610 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.631 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.643 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.658 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.672 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.689 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.702 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.718 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.730 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.746 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.760 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.776 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.799 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.816 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.830 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.848 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.859 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.875 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.891 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.909 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.206.921 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.940 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.977 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.206.997 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.012 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.032 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.050 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.071 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.089 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.105 [hccl_communicator_attrs.cc:156] [619756][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.124 [hccl_communicator_attrs.cc:1021] [619756][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.144 [hccl_communicator.cc:1440] [619756][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.157 [hccl_communicator.cc:1450] [619756][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.170 [externalinput.cc:1445] [619756]InitParsesProtocol start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.183 [externalinput.cc:1462] [619756]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:48.207.197 [externalinput.cc:1411] [619756]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.221 [externalinput.cc:1413] [619756]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.236 [externalinput.cc:1467] [619756]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.280 [config.cc:557] [619756][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.299 [hccl_communicator.cc:668] [619756][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.326 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.346 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.361 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=0. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.385 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=0.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.399 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.412 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.427 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.442 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=1. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.462 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=1.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.476 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.487 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.501 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.517 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=2. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.532 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=2.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.544 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.556 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.572 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.587 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=3. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.611 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=3.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.624 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.639 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.657 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.672 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=4. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.690 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=4.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.705 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.718 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.731 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.745 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=6. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.760 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=6.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.776 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.793 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.810 [api_impl.cc:3520] 619756 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.207.824 [api_impl.cc:3168] 619756 EnableP2P: Enable P2P devIdDes=5, phyIdSrc=7. [INFO] DRV(619439,python3.9):2025-04-03-09:35:48.207.839 [devdrv_pcie.c:888][ascend][curpid:619439,619756][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=5; peer_phy_devid=7.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.852 [p2p_mgmt.cc:74] [619756]enable p2p: local logic id:5, remote physic id:7. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.875 [adapter_hal.cc:298] [619756]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.895 [sal.cc:593] [619756]IsHostUseDevNic[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.941 [adapter_hal.cc:298] [619756]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.959 [sal.cc:593] [619756]IsHostUseDevNic[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.969 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.207.983 [network_manager.cc:169] [619756]NetworkManager: devicePhyId[4294967295], devicePhyId_[5] deviceLogicId_[5], hasBackup[0] [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.000 [tsd_client.cpp:189][TsdCapabilityGet][tid:619756] TsdCapabilityGet Begin. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.014 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.027 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.042 [process_mode_manager.cpp:1042][CapabilityGet][tid:619756] [ProcessModeManager] enter into CapabilityGet process deviceId[5] type[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.208.059 [adapter_tdt.cc:66] [619756]Get TsdCapability success. deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.208.070 [network_manager.cc:175] [619756][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:48.208.094 [network_manager.cc:99] [619756][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.110 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.122 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619756] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.132 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619756] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.145 [process_mode_manager.cpp:386][InitTsdClient][tid:619756] [TsdClient] tsd client has been inited before [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.171 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619756] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619439,python3.9):2025-04-03-09:35:48.208.201 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619756] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.213 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619756] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.226 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619756] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.245 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619756] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.208.286 [hdc_client.cpp:154][TsdRecvData][tid:619756] recv timeout is:150000 ms [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.388.179 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.388.179 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.388.203 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.388.213 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.223 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.229 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.239 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.244 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.388.253 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.388.280 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.388.286 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.388.293 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.303 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.388.307 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.704.178 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.704.178 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.299 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.319 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.332 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.332 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.346 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.354 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.360 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.369 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.374 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.704.381 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.393 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.704.407 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.708.169 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.708.171 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.708.175 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.708.177 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.265 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.281 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.290 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.295 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.308 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.318 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.320 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.327 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.335 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.342 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.346 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.349 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.358 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.363 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.370 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.378 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.390 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.396 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.404 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.708.419 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.414 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.434 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.439 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.708.446 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.900.164 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.900.173 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.900.201 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:48.900.208 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.221 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.227 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.238 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.244 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.900.249 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.900.256 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.900.261 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:48.900.269 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.276 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.900.286 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:48.940.720 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.940.754 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.940.773 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:48.940.786 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.412.171 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.412.172 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.412.196 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.412.203 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.211 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.222 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.227 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.236 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.412.241 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.412.260 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.412.267 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.412.275 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.283 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.412.294 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.171 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.171 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.172 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.177 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.177 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.540.178 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.277 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.297 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.301 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.311 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.316 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.322 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.329 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.332 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.333 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.342 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.348 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.351 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.357 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.360 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.382 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.393 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.405 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.415 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.397 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.430 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.433 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.439 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.483 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.467 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.472 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.490 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.501 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.513 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.507 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.530 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.538 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.553 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.540.544 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.567 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.577 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.540.613 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.924.173 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.924.177 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.924.213 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:49.924.221 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.236 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.240 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.251 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.256 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.924.267 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.924.272 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.924.279 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:49.924.285 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.296 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:49.924.301 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] TDT(619439,python3.9):2025-04-03-09:35:50.311.240 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619756] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.311.248 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:50.311.280 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619756] [TsdClient] DeviceMsgProc recvMsg realDeviceId[5] msgType[49] localDevId[5] rspCode[0]heterogeneousSubPid[31150], tsdSupportLevel_[63] [INFO] TDT(619439,python3.9):2025-04-03-09:35:50.311.299 [hdc_client.cpp:172][TsdRecvData][tid:619756] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:50.311.315 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619756] OpenSubProc success stype:0, pid:31150 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.311.332 [adapter_tdt.cc:39] [619756]Open TsdProcess success. deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.311.347 [network_manager.cc:118] [619756][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31150], devicePhyId_[5], deviceLogicId_[5], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.311.363 [network_manager.cc:180] [619756][NetworkManager][Init]open tsd by process success, devicePhyId[5], deviceLogicId_[5]. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.311.376 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:50.311.433 [ra_host.c:205]tid:619756,ra_init(205) : Input parameters: phy_id[5], nic_position:[1] hdc_type:[18] [INFO] ROCE(619439,python3.9):2025-04-03-09:35:50.311.545 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619439,python3.9):2025-04-03-09:35:50.311.573 [devdrv_hccl.c:103][ascend][curpid:619439,619756][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619439) [INFO] ROCE(619439,python3.9):2025-04-03-09:35:50.311.639 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:50.311.657 [ra_hdc.c:580]tid:619756,ra_hdc_init(580) : hdc init start! logic id is 5, phy id is 5, hdc_type is 18 [INFO] DRV(619439,python3.9):2025-04-03-09:35:50.311.828 [hdc_core.c:1982][ascend][curpid:619439,619756][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4276; pid=619439) [INFO] HCCP(619439,python3.9):2025-04-03-09:35:50.312.081 [ra_hdc.c:616]tid:619756,ra_hdc_init(616) : hdc init OK! phy_id[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.315.952 [adapter_hccp.cc:583] [619756]init ra success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.315.972 [network_manager.cc:222] [619756]NetworkManager nicDeploy[1] deviceLogicId_[5] devicePhyId_[5] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.315.993 [hccl_communicator_attrs.cc:1307] [619756][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.005 [hccl_communicator_attrs.cc:1275] [619756]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.025 [hccl_communicator.cc:2259] [619756][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.053 [adapter_hccp.cc:1650] [619756]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.303 [adapter_hccp.cc:2343] [619756]Get available Vnic info success, deviceIdType[0] deviceId[0x5] Vnic ip[192.168.2.194] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.353 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:50.316.378 [ra_host.c:280]tid:619756,ra_socket_init(280) : socket init:mode=1 phy_id=5 family=2 ip=192.168.2.194 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.394 [adapter_hccp.cc:806] [619756]socket init success, ip[3254954176] device id[5], socketHandle[0xfffe7c0ffbb0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.409 [network_manager.cc:692] [619756]devicePhyId[5], ip[192.168.2.194] socket init OK [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.419 [network_manager.cc:432] [619756][Start][Vnic]trying to listen on ip[192.168.2.194] port[16666]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:50.316.439 [ra_host.c:875]tid:619756,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[5], local_ip[192.168.2.194], port[16666] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.749 [network_manager.cc:448] [619756][Start][Vnic]Listen on ip[192.168.2.194], port[16666] sucess, devPhyId[5], devLogicId[5], isAutoPort[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.768 [network_manager.cc:451] [619756]NetworkManager devicePhyId_[5] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.784 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.797 [hccl_socket.cc:106] [619756][HcclSocket][Listen] device[5] listens on ip[192.168.2.194] port[16666] success, socketType[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.810 [hccl_socket_manager.cc:56] [619756][Init][Server]ip[192.168.2.194] port[16666] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.829 [hccl_communicator.cc:1548] [619756][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.194] port[16666] success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.844 [hccl_communicator_attrs.cc:1307] [619756][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.858 [hccl_communicator_attrs.cc:1275] [619756]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.892 [hccl_communicator.cc:1565] [619756]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.905 [network_manager.cc:69] [619756]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.316.919 [hccl_communicator.cc:2028] [619756]profiling config information:options[], mode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.052 [adapter_trace.cc:21] [619756]Call TraceCreate, return value[0] [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.317.071 [atrace_api.c:61](tid:619756) AtraceCreateWithAttr start. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.317.145 [trace_rb_log.c:45](tid:619756) init ring buffer time finished, realTime 1743644150317138782, monotonicTime 4211952707138, freq 99993 kHz, minutesWest 480 minutes [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.317.164 [trace_rb_log.c:137](tid:619756) [HCCL_619756_5] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.317.183 [tracer_mgr_operate.c:66](tid:619756) create object HCCL_619756_5 success, exitSave(true), noLock(false). [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.317.195 [atrace_api.c:63](tid:619756) AtraceCreateWithAttr end. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.207 [adapter_trace.cc:70] [619756]Call TraceCreateWithAttr, return value[0xaaab07266908], Params: objName[HCCL_619756_5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.249 [rank_consistentcy_checker.cc:32] [619756][GetInstance] get deviceLogicId[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.298 [api_impl.cc:1486] 619756 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.325 [npu_driver.cc:4604] 619756 GetMaxStreamAndTask: deviceId=5, tsId=0, streamCount=1984. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.343 [task_exception_handler.cc:1212] [619756]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.364 [hccl_communicator_attrs.cc:1099] [619756][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.422 [api_impl.cc:4219] 619756 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.445 [command_handle.cc:25] [619756][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.460 [command_handle.cc:99] [619756][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.573 [profiler_manager_impl.cc:76] [619756][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.589 [hccl_communicator.cc:2193] [619756][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.317.603 [hccl_communicator.cc:803] [619756]Will use ffts mode. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.632 [api_impl.cc:3138] 619756 GetDeviceIndexByPhyId: get Index by PhyId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.651 [api_impl.cc:3149] 619756 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.664 [api_impl.cc:3157] 619756 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.317.673 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.319.144 [api_impl.cc:3476] 619756 GetDeviceInfo: get device info, deviceId=5 [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:50.319.201 [tbe_vector_reduce.cc:508] [619756]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.216 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.390 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.320.413 [dispatcher.cc:157] [619756][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.432 [api_impl.cc:3138] 619756 GetDeviceIndexByPhyId: get Index by PhyId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.445 [api_impl.cc:3149] 619756 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.457 [api_impl.cc:3157] 619756 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.471 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.320.496 [api_impl.cc:1990] 619756 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.321.329 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.321.396 [api_impl.cc:1990] 619756 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.321.423 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.321.457 [api_impl.cc:1990] 619756 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.321.475 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.504 [opexecounter.cc:71] [619756]alloc counter mem resource. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.550 [hccl_communicator_attrs.cc:1368] [619756][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.604 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.628 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.653 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.667 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.683 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.695 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.707 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.721 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.735 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.749 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.780 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.794 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.810 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.821 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.833 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.847 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.860 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.872 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.885 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.899 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.910 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.922 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.934 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.948 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.961 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.973 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.985 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.321.999 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.012 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.022 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.034 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.048 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.059 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.078 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.092 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.106 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.120 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.133 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.146 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.161 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.172 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.184 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.198 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.212 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.224 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.236 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.249 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.263 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.278 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.292 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.303 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.319 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.334 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.347 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.362 [alg_configurator.cc:68] [619756][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.383 [alg_configurator.cc:241] [619756][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.395 [alg_configurator.cc:134] [619756]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.422 [alg_configurator.cc:449] [619756][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.437 [alg_configurator.cc:42] [619756]alg configurator init success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.476 [topo_info_extractor.cc:66] [619756]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[5], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.500 [topo_info_extractor.cc:155] [619756][SetRankInfo]rankData_: userRank[5], devicePhyId[5], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.527 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.561 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.594 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.614 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.633 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.653 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.666 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.681 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.717 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.736 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.751 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.766 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.322.780 [api_error.cc:3478] 619756 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.796 [adapter_rts.cc:940] [619756][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.815 [topo_info_extractor.cc:348] [619756]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.322.979 [topo_info_extractor.cc:888] [619756]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[5], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.013 [topo_info_extractor.cc:585] [619756][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.046 [topo_info_extractor.cc:646] [619756][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.065 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.082 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.096 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.111 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.124 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.141 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/1; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.155 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.169 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.182 [topo_info_extractor.cc:689] [619756]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[5], userRankSize[8], plane size[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.194 [topo_info_extractor.cc:697] [619756][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.207 [topo_info_extractor.cc:734] [619756]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 5/5/0/; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.221 [topo_info_extractor.cc:739] [619756]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[5], userRankSize[8], plane size[1] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.232 [topo_info_extractor.cc:585] [619756][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.249 [topo_info_extractor.cc:646] [619756][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.264 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.284 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.304 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.320 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.336 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.363 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/1; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.379 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.394 [topo_info_extractor.cc:685] [619756]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.407 [topo_info_extractor.cc:689] [619756]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[5], userRankSize[8], plane size[8] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.452 [topo_info_extractor.cc:790] [619756]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId5/5; ] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.473 [topo_info_extractor.cc:519] [619756]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.323.615 [api_impl.cc:1990] 619756 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.323.645 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.704 [hccl_impl.cc:59] [619756]start hccl resources build:get deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.323.847 [hccl_impl.cc:177] [619756]hcclImpl init success. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:50.323.880 [hccl_aiv.cc:279] [619756][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.326.939 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.019 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.130 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.151 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.169 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.189 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.212 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.310 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.327 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.342 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.357 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.376 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.480 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.506 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.522 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.538 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.559 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.659 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.675 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.690 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.704 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.724 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.817 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.833 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.848 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.861 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.879 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.978 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.327.994 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.008 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.021 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.042 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.137 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.153 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.186 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.203 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.223 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.319 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.335 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.360 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.375 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.393 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.488 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.506 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.520 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.533 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.552 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.647 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.663 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.681 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.694 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.712 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.806 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.823 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.837 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.849 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.866 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.969 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.328.986 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.000 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.013 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.034 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.127 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.143 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.156 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.170 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.197 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.289 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.305 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.320 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.334 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.353 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.451 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.467 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.481 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.495 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.514 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.608 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.625 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.641 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.655 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.673 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.766 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.783 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.796 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.812 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.830 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.924 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.940 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.954 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.969 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.329.989 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.093 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.110 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.125 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.139 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.157 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.248 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.264 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.278 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.292 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.311 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.404 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.419 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.431 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.446 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.463 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.558 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.575 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.591 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.605 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.624 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.719 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.735 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.751 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.766 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.787 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.880 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.897 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.925 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.941 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.330.961 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.053 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.071 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.085 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.099 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.117 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.208 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.225 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.239 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.254 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.272 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.370 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.386 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.403 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.420 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.438 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.533 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.551 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.565 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.579 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.597 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.689 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.705 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.719 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.742 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.761 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.855 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.873 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.888 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.902 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.331.922 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.018 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.034 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.051 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.066 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.084 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.185 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.209 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.226 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.243 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.261 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.354 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.370 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.385 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.400 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.417 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.513 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.530 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.545 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.559 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.593 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.689 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.705 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.720 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.735 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.752 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.849 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.867 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.881 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.895 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.332.913 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.003 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.022 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.036 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.049 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.066 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.160 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.179 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.193 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.207 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.224 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.318 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.335 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.349 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.363 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.380 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.480 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.505 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.519 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.534 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.551 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.645 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.662 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.677 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.691 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.709 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.802 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.820 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.835 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.849 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.867 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.958 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.974 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.333.989 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.002 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.020 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.117 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.133 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.148 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.162 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.180 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.273 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.290 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.312 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.326 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.344 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.436 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.453 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.467 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.480 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.499 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.591 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.608 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.622 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.635 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.652 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.746 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.762 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.778 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.792 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.811 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.908 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.924 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.939 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.955 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.334.977 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.068 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.084 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.098 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.121 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.140 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.233 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.249 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.266 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.279 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.299 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.391 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.407 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.424 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.442 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.463 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.556 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.574 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.588 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.601 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.619 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.717 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.734 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.749 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.761 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.779 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.874 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.893 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.909 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.924 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.335.958 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.054 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.072 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.090 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.104 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.125 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.233 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.260 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.276 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.291 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.311 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.402 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.420 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.436 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.451 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.469 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.556 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.576 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.594 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.607 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.624 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.720 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.739 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.754 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.772 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.789 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.892 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.917 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.933 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.946 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.336.963 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.059 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.076 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.092 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.107 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.126 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.226 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.244 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.258 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.272 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.291 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.394 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.411 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.425 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.439 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.456 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.548 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.565 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.579 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.596 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.616 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.706 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.724 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.738 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.763 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.783 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.874 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.892 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.906 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.922 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.337.942 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.041 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.058 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.072 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.085 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.104 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.194 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.212 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.227 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.243 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.261 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.354 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.372 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.386 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.401 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.421 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.513 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.530 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.544 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.560 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.586 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.686 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.705 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.720 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.734 [runtime.cc:1673] 619756 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.754 [elf.cc:566] 619756 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.845 [runtime.cc:1688] 619756 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.861 [runtime.cc:1953] 619756 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.875 [runtime.cc:1897] 619756 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619439,python3.9):2025-04-03-09:35:50.338.909 [devmm_svm.c:1972][ascend][curpid:619439,619756][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.338.924 [hdc.cc:83] [619756][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.939 [api_impl.cc:1990] 619756 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.338.965 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.536 [api_impl.cc:1990] 619756 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.559 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.580 [api_impl.cc:2664] 619756 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619439,python3.9):2025-04-03-09:35:50.339.733 [devmm_svm.c:1972][ascend][curpid:619439,619756][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.339.752 [hdc.cc:83] [619756][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.768 [api_impl.cc:1990] 619756 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.786 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.814 [api_impl.cc:2096] 619756 HostMalloc: size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.339.833 [api_error.cc:1260] 619756 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.026 [api_impl.cc:1990] 619756 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.098 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.120 [api_impl.cc:2664] 619756 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.282 [api_impl.cc:1990] 619756 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.318 [api_error.cc:1218] 619756 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.336 [api_impl.cc:2664] 619756 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.370 [peterson_lock.cc:69] [619756][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.387 [peterson_lock.cc:47] [619756][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.410 [hccl_comm.cc:129] [619756]hcclCommInitInfo:commId[hccl-619439-0-1743644148-hccl_world_group], rank[5], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[5], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.434 [hccl_comm.cc:175] [619756][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.451 [topo_matcher.cc:528] [619756][SetAivMode]AivMode is set to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.464 [topo_matcher.cc:540] [619756][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.484 [hcom_plugin.cc:349] [619756][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.340.527 [runtime.cc:5239] 619756 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.542 [adapter_rts.cc:198] [619756][hrtGetDeviceRefresh]deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.555 [hcom_common.cc:85] [619756][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.585 [op_base.cc:128] [619756][HcclGetDeviceId] deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.603 [hccl_comm.cc:1296] [619756]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffe83ffccb8] topoSize[2] [INFO] GE(619439,python3.9):2025-04-03-09:35:50.340.630 [hcom_topo_info.cc:24]619756 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.646 [hcom_plugin.cc:371] [619756][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.673 [topoinfo_detect.cc:555] [619756]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.687 [topoinfo_detect.cc:321] [619756]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.700 [topoinfo_detect.cc:219] [619756]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.722 [op_base.cc:1050] [619756][InitCommRootInfo]Init success, rankNum[8], rank[5], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.741 [profiler_base.cc:342] [619756]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.757 [op_base.cc:1107] [619756][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4980947]us, rankNum[8], rank[5] [WARNING] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.340.824 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.340.856 [op_base.cc:853] [619756]HcclGetCommName input handle=0xfffe7c022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.340.898 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 5 [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.340.923 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 5, group name: hccl_world_group [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.341.026 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.341.049 [device.cpp:183]619756 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 5. [INFO] DEVICE(619439,fffe82fdefa0,python3.9):2025-04-03-09:35:50.341.059 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 5, group name: hccl_world_group [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.341.073 [api_impl.cc:3419] 619756 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.341.088 [runtime.cc:3444] 619756 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.341.103 [api_impl.cc:3449] 619756 DeviceReset: Succ, userDeviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.341.115 [device.cpp:189]619756 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 5. [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.341.135 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:50.341.225 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5282.9 msec. [WARNING] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:50.341.270 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.341.350 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 8285.61 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.341.387 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 8285.67 msec. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.341.407 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.341.432 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9385.1 msec. [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.341.469 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.148 [device.cpp:143]619439 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.203 [api_impl.cc:2999] 619439 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.221 [device.cc:306] 619439 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.239 [runtime.cc:3393] 619439 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.257 [runtime.cc:3560] 619439 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.271 [api_impl.cc:2987] 619439 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.286 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.303 [api_impl.cc:2992] 619439 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.322 [api_impl.cc:3025] 619439 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.340 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.354 [acl.cpp:121]619439 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.374 [device.cpp:154]619439 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.354.419 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.442 [context.cpp:174]619439 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.354.465 [api_impl.cc:6913] 619439 CtxSetSysParamOpt: curCtx = 0xaaab18d025e0, configOpt=0, configVal=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.354.483 [context.cpp:158]619439 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.355.090 [op_base.cc:1214] [619439][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.355.113 [op_base.cc:128] [619439][HcclGetDeviceId] deviceLogicId[5] [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.355.138 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.355.157 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.355.173 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.355.214 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.355.851 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.345 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.425 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.613 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 193.75 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.651 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.707 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 52.92 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.733 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.754 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 21.75 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.773 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.841 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 65.24 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.883 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.538 msec. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.356.986 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.050 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 7.72 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.129 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 49.08 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.175 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.185 msec. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.208 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.249 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.303 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 52.37 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.328 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.365 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 34.97 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.388 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.407 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 18.22 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.424 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.459 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 31.92 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.514 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.302 msec. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.550 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.675 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.762 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 85.54 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.826 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 36.55 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.874 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 19.54 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 31.63 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.357.998 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 41.35 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.144 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 116.92 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.173 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 2.37 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.194 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.270 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 55.45 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 29.76 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.350 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.422 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 49.76 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.358.981 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 527.731 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.008 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 37.71 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.124 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.569 msec. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.163 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.219 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 2.232 msec. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.655 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 66.971 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.685 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.773 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 84.69 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.798 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.865 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 63.94 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.888 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.922 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 32.25 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.963 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.359.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 43.66 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.015 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 7.21 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.069 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.092 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 53.68 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.110 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.127 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 17.34 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.144 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.174 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 28.59 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.233 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.266 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 63.14 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.286 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.304 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 17.7 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.324 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.345 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 19.651 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.363 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.409 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 40.35 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.480 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.504 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 68.19 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.523 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.544 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 20.95 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.562 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.580 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 16.43 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.601 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.617 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 15.77 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.636 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.652 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 15.33 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.689 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.711 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 39.52 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.733 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.800 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 64.3 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.850 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.873 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 41.01 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.893 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 15.96 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.930 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 17 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.969 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.360.988 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 19.04 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.010 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.028 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 18.11 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.069 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.090 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 41.64 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.123 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.144 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 35.06 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.162 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.183 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 20.9 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.202 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.223 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 19.44 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.325 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 78.4 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.627 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 273.06 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.653 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.725 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 36.401 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.749 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.820 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 49.55 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.843 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.891 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 27.99 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.911 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.965 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 32.8 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.361.985 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.026 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 22.94 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.048 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.140 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 70.94 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.162 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.227 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 46.8 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.253 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.373 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 96.46 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.397 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.473 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 55.49 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.493 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.528 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.663 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 135.7 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.709 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.836 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 126.02 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.859 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.953 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 71.97 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.362.976 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.056 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 58.621 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.081 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.102 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.121 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.151 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.164 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.184 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.199 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.214 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.248 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.262 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.277 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.292 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.308 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.324 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.578 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 232.14 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.606 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.625 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.641 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.657 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.671 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.772 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 75.981 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.795 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.814 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.830 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.916 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 64.25 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.363.977 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.703 msec. [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.364.344 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.364.498 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.364.820 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 251.52 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.364.959 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 36.77 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.075 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 83.15 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 17.17 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.172 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 28.45 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.286 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 86.81 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.340 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 28.54 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.391 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 24.45 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.483 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 65.62 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.526 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 17.32 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.577 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.685 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.637 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.139 msec. [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.365.683 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.120 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.158 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.491 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.183 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.203 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.356 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 65.09 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.456 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 69.01 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.528 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 44.44 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.624 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 69.97 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.648 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.666 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.726 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 39.44 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.766 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 15.19 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.791 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 1.85 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.836 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.594 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.894 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.653 msec. [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.366.935 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.050 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.076 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 1.25 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.129 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 31.19 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.158 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 4.95 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.180 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.55 us [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.214 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.292 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 89.74 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.326 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 1.38 us [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.354 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.335 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.411 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.491 msec. [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.449 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.367.530 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.091 msec. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.368.060 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.368.092 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.368.111 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.153 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.440 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.489 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.599 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.706 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.12 msec. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.803 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.862 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.889 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.908 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.925 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.368.946 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.369.002 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.369.256 [context.cpp:72]619791 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.369.294 [api_impl.cc:3717] 619791 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.369.316 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.369.319 [awatchdog_core.c:175](tid:619791) create watchdog for id : 65543, tid : 619791, timeout : 300s [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.369.339 [context.cpp:80]619791 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.369.389 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.369.434 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.369.459 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61027123200, align_size:61027123200 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.370.827 [api_impl.cc:7115] 619791 ReserveMemAddress: device malloc Succ, size=61027123200(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5800000 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.175 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.175 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.185 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.188 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.188 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.372.190 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.298 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.317 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.333 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.333 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.355 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.369 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.352 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.382 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.382 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.416 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.434 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.416 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.440 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.453 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.472 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.460 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.463 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.477 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.486 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.498 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.514 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.525 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.532 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.535 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.550 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.570 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.581 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.585 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.590 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.602 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.616 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.372.640 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.652 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.660 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.685 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.372.695 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.379.130 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.170 [api_impl.cc:7129] 619791 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.296 [api_impl.cc:7145] 619791 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.376 [device.cpp:143]619791 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.399 [api_impl.cc:2999] 619791 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.411 [device.cc:306] 619791 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.426 [runtime.cc:3393] 619791 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.444 [runtime.cc:3560] 619791 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.459 [api_impl.cc:2987] 619791 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.474 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.491 [api_impl.cc:2992] 619791 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.509 [api_impl.cc:3025] 619791 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.524 [acl.cpp:115]619791 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.537 [acl.cpp:121]619791 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.550 [device.cpp:154]619791 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.379.578 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.597 [context.cpp:174]619791 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.615 [api_impl.cc:6913] 619791 CtxSetSysParamOpt: curCtx = 0xaaab18d025e0, configOpt=0, configVal=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.628 [context.cpp:158]619791 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.379.685 [op_base.cc:1214] [619791][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.706 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.379.722 [adapter_rts.cc:234] [619791][hrtGetDevice]deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:50.379.734 [op_base.cc:128] [619791][HcclGetDeviceId] deviceLogicId[5] [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.768 [context.cpp:72]619791 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.798 [api_impl.cc:3717] 619791 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.379.813 [context.cpp:80]619791 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.839 [api_error.cc:1807] 619791 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.859 [memory_task.cc:342] 619791 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.874 [memory_task.cc:473] 619791 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.885 [memory_task.cc:518] 619791 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.905 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.922 [memory_task.cc:1007] 619791 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaab239ea870, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.937 [memory_task.cc:1166] 619791 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.953 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.966 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.379.988 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.002 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.380.025 [callback.cpp:97]619791 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.055 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.078 [task_info.cc:521] 619791 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.093 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.107 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.121 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.134 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.380.149 [callback.cpp:109]619791 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.380.243 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.380.275 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.267 [npu_driver.cc:3844] 619548 CqReportGet: cq report get start, device_id=5, tsId=0. [INFO] DEVICE(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.380.309 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61027123200, align_size:61027123200 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.314 [npu_driver.cc:3851] 619548 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.328 [api_impl.cc:5087] 619548 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.380.350 [callback.cpp:138]619548 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.380.367 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.380.386 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.381.594 [api_impl.cc:7115] 619791 ReserveMemAddress: device malloc Succ, size=61027123200(bytes) start ptr=0x12d100000000, end ptr=0x12df35800000 [INFO] PRE_ACT(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.390.408 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.445 [api_impl.cc:7129] 619791 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.551 [api_impl.cc:7145] 619791 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.390.664 [device.cpp:231]619791 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.687 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.390.705 [acl.cpp:802]619791 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.823 [npu_driver.cc:404] 619791 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.863 [stream.cc:2246] 619791 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.902 [stream_sqcq_manage.cc:147] 619791 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.390.919 [npu_driver.cc:882] 619791 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.510 [npu_driver.cc:6125] 619791 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.565 [npu_driver.cc:6137] 619791 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=371, streamId=6, logicCq=371, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.593 [npu_driver.cc:6189] 619791 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=6, logicCq=371. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.616 [npu_driver.cc:5916] 619791 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c4047000, addr=dfffc4047000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.631 [stream.cc:653] 619791 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.686 [stream.cc:276] 619791 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.715 [stream.cc:688] 619791 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=5, streamResId=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.391.733 [acl_resource_manager.cpp:814]619791 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.391.748 [acl_resource_manager.cpp:816]619791 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.906 [engine.cc:188] 619791 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.929 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.950 [task_info.cc:409] 619791 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.966 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.979 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.391.996 [npu_driver.cc:751] 619791 SqTaskSend: device_id=5, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.012 [stars_engine.cc:893] 619791 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.031 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.048 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.067 [stream.cc:2886] 619791 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.082 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.096 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.112 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.125 [logger.cc:152] 619791 TaskFinished: device_id=5, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.147 [stars_engine.cc:437] 619791 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.177 [stream.cc:2886] 619791 StarsWaitForTask: No.1 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.199 [stream.cc:852] 619791 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.218 [stream.cc:938] 619791 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.230 [stream.cc:1507] 619791 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.256 [stream_sqcq_manage.cc:451] 619791 FreeLogicCq: Return(0), threadIdentifier(281468601364384), devId(5), tsId(0), cqId(371), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.277 [npu_driver.cc:6163] 619791 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=6, logicCq=371. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.312 [npu_driver.cc:1180] 619791 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=371, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.330 [stream.cc:237] 619791 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.366 [stream_sqcq_manage.cc:216] 619791 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.392.381 [npu_driver.cc:966] 619791 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.393.317 [npu_driver.cc:974] 619791 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.393.337 [stream_sqcq_manage.cc:228] 619791 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.393.354 [npu_driver.cc:432] 619791 StreamIdFree: stream id free begin, device_id=5, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.393.375 [npu_driver.cc:441] 619791 StreamIdFree: Stream id free success, stream_id=6, device_id=5, tsId=0, streamFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.393.394 [device_msg_handler.cc:110] 619791 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.394.147 [device_msg_handler.cc:127] 619791 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.394.196 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.394.220 [npu_driver.cc:500] 619791 GetFaultEvent: get fault event, device_id=5, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.394.306 [acl.cpp:782]619791 GetFaultEventInfo: can not get fault event of device 5, runtime errorCode is 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:50.394.336 [error_manager.cc:989]619791 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.394.684 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.394.725 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.394.749 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.394.769 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.395.133 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.230 [context.cpp:72]619791 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.255 [api_impl.cc:3717] 619791 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.272 [context.cpp:80]619791 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.395.282 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.292 [api_error.cc:1807] 619791 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.308 [memory_task.cc:342] 619791 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.331 [memory_task.cc:473] 619791 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.346 [memory_task.cc:518] 619791 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.358 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.372 [memory_task.cc:1007] 619791 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaab239eb290, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.390 [memory_task.cc:1166] 619791 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.402 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.413 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.430 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.444 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.462 [callback.cpp:97]619791 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.479 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.495 [task_info.cc:521] 619791 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.508 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.523 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.538 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.552 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.563 [callback.cpp:109]619791 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.583 [npu_driver.cc:3844] 619548 CqReportGet: cq report get start, device_id=5, tsId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.605 [npu_driver.cc:3851] 619548 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.619 [api_impl.cc:5087] 619548 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.631 [callback.cpp:138]619548 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.639 [device.cpp:231]619791 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.650 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.659 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.666 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.395.683 [acl.cpp:802]619791 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.714 [npu_driver.cc:404] 619791 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.740 [stream.cc:2246] 619791 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.780 [stream_sqcq_manage.cc:147] 619791 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.395.798 [npu_driver.cc:882] 619791 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.053 [npu_driver.cc:6125] 619791 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.097 [npu_driver.cc:6137] 619791 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=372, streamId=7, logicCq=372, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.120 [npu_driver.cc:6189] 619791 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=7, logicCq=372. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.135 [npu_driver.cc:5916] 619791 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c4047000, addr=dfffc4047000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.150 [stream.cc:653] 619791 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.205 [stream.cc:276] 619791 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.221 [stream.cc:688] 619791 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=5, streamResId=65534 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.396.238 [acl_resource_manager.cpp:814]619791 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.396.251 [acl_resource_manager.cpp:816]619791 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.292 [engine.cc:188] 619791 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.311 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.331 [task_info.cc:409] 619791 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.346 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.359 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.375 [npu_driver.cc:751] 619791 SqTaskSend: device_id=5, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.390 [stars_engine.cc:893] 619791 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.405 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.421 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.434 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.460 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.473 [logger.cc:152] 619791 TaskFinished: device_id=5, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.488 [stars_engine.cc:437] 619791 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.503 [stream.cc:2886] 619791 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.515 [stream.cc:852] 619791 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.528 [stream.cc:938] 619791 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.540 [stream.cc:1507] 619791 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.558 [stream_sqcq_manage.cc:451] 619791 FreeLogicCq: Return(0), threadIdentifier(281468601364384), devId(5), tsId(0), cqId(372), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.578 [npu_driver.cc:6163] 619791 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=7, logicCq=372. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.604 [npu_driver.cc:1180] 619791 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=372, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.618 [stream.cc:237] 619791 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.631 [stream_sqcq_manage.cc:216] 619791 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.396.643 [npu_driver.cc:966] 619791 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.231 [npu_driver.cc:974] 619791 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.252 [stream_sqcq_manage.cc:228] 619791 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.267 [npu_driver.cc:432] 619791 StreamIdFree: stream id free begin, device_id=5, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.286 [npu_driver.cc:441] 619791 StreamIdFree: Stream id free success, stream_id=7, device_id=5, tsId=0, streamFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.301 [device_msg_handler.cc:110] 619791 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.647 [device_msg_handler.cc:127] 619791 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.680 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.397.701 [npu_driver.cc:500] 619791 GetFaultEvent: get fault event, device_id=5, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.397.765 [acl.cpp:782]619791 GetFaultEventInfo: can not get fault event of device 5, runtime errorCode is 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:50.397.784 [error_manager.cc:989]619791 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.398.073 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.398.106 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.398.129 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.398.158 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.398.229 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.314 [context.cpp:72]619791 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.336 [api_impl.cc:3717] 619791 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.351 [context.cpp:80]619791 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.371 [api_error.cc:1807] 619791 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.389 [memory_task.cc:342] 619791 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.402 [memory_task.cc:473] 619791 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.414 [memory_task.cc:518] 619791 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.433 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.452 [memory_task.cc:1007] 619791 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaab239ebd00, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.466 [memory_task.cc:1166] 619791 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.480 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.494 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.511 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.528 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.545 [callback.cpp:97]619791 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.565 [task_res.cc:380] 619791 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.580 [task_info.cc:521] 619791 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.597 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.611 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.636 [task_submit.cc:573] 619791 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.651 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.664 [callback.cpp:109]619791 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.669 [npu_driver.cc:3844] 619548 CqReportGet: cq report get start, device_id=5, tsId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.685 [npu_driver.cc:3851] 619548 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.698 [api_impl.cc:5087] 619548 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.712 [callback.cpp:138]619548 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.724 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.730 [device.cpp:231]619791 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.736 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.747 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.398.761 [acl.cpp:802]619791 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.795 [npu_driver.cc:404] 619791 StreamIdAlloc: stream id alloc begin, device_id=5, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.823 [stream.cc:2246] 619791 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.863 [stream_sqcq_manage.cc:147] 619791 Alloc: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.398.879 [npu_driver.cc:882] 619791 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.123 [npu_driver.cc:6125] 619791 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.172 [npu_driver.cc:6137] 619791 LogicCqAllocateV2: success: deviceId=5, tsId=0, cqId=373, streamId=8, logicCq=373, cqeDepth=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.195 [npu_driver.cc:6189] 619791 StreamBindLogicCq: success: deviceId=5, tsId=0, streamId=8, logicCq=373. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.212 [npu_driver.cc:5916] 619791 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=c4047000, addr=dfffc4047000size=4096 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.227 [stream.cc:653] 619791 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.271 [stream.cc:276] 619791 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.287 [stream.cc:688] 619791 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=5, streamResId=65534 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.399.304 [acl_resource_manager.cpp:814]619791 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.399.315 [acl_resource_manager.cpp:816]619791 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.361 [engine.cc:188] 619791 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.392 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.408 [task_info.cc:409] 619791 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.422 [stream.cc:1400] 619791 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.436 [stream.cc:2595] 619791 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.453 [npu_driver.cc:751] 619791 SqTaskSend: device_id=5, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.465 [stars_engine.cc:893] 619791 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.481 [stream.cc:1494] 619791 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.494 [npu_driver.cc:5547] 619791 GetSqHead: deviceId=5, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.509 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.524 [task.cc:185] 619791 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.536 [logger.cc:152] 619791 TaskFinished: device_id=5, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.552 [stars_engine.cc:437] 619791 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.566 [stream.cc:2886] 619791 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.578 [stream.cc:852] 619791 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.593 [stream.cc:938] 619791 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.604 [stream.cc:1507] 619791 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.626 [stream_sqcq_manage.cc:451] 619791 FreeLogicCq: Return(0), threadIdentifier(281468601364384), devId(5), tsId(0), cqId(373), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.648 [npu_driver.cc:6163] 619791 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=8, logicCq=373. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.677 [npu_driver.cc:1180] 619791 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=373, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.694 [stream.cc:237] 619791 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.708 [stream_sqcq_manage.cc:216] 619791 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.399.720 [npu_driver.cc:966] 619791 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.196 [npu_driver.cc:974] 619791 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.218 [stream_sqcq_manage.cc:228] 619791 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.236 [npu_driver.cc:432] 619791 StreamIdFree: stream id free begin, device_id=5, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.267 [npu_driver.cc:441] 619791 StreamIdFree: Stream id free success, stream_id=8, device_id=5, tsId=0, streamFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.284 [device_msg_handler.cc:110] 619791 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.636 [device_msg_handler.cc:127] 619791 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.668 [runtime.cc:5239] 619791 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.400.691 [npu_driver.cc:500] 619791 GetFaultEvent: get fault event, device_id=5, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.400.747 [acl.cpp:782]619791 GetFaultEventInfo: can not get fault event of device 5, runtime errorCode is 0 [INFO] GE(619439,python3.9):2025-04-03-09:35:50.400.766 [error_manager.cc:989]619791 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.401.052 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.401.084 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.401.106 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.401.123 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619439,fffe83ffefa0,python3.9):2025-04-03-09:35:50.401.194 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:50.406.761 [awatchdog_monitor.c:160](tid:619472) thread 619756 not exist [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.436.172 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.436.208 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.436.228 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.436.251 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.436.271 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.436.289 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.436.305 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.916.185 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.916.226 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.916.249 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.916.287 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.916.307 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.916.322 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.916.339 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.948.169 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:50.948.206 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.948.221 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:50.948.238 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.948.255 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.948.274 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.948.292 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.977.980 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.021 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.978.041 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.978.084 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.106 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.128 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.193 [stars_engine.cc:643] 619439 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.223 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.244 [task_manager.cc:485] 619439 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.261 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.279 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.297 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.318 [stars_engine.cc:643] 619439 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.348 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.364 [task_manager.cc:485] 619439 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.376 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.392 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.406 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.425 [stars_engine.cc:643] 619439 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.440 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.454 [task_manager.cc:485] 619439 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.467 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.486 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.504 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.535 [stars_engine.cc:622] 619439 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=367, retCode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.553 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.568 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.978.581 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.978.602 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.620 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.635 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.651 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.668 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.680 [stream.cc:2886] 619439 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.693 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.706 [stream.cc:2886] 619439 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.718 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.732 [stream.cc:2886] 619439 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.762 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.774 [stream.cc:2886] 619439 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.790 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.806 [stream.cc:2886] 619439 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.820 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.833 [stream.cc:2886] 619439 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.846 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.860 [stream.cc:2886] 619439 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.875 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.887 [stream.cc:2886] 619439 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.900 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.913 [stream.cc:2886] 619439 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.929 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.945 [stream.cc:2886] 619439 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.960 [event.cc:310] 619439 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.978.986 [task_res.cc:380] 619439 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=5, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.011 [event.cc:249] 619439 UpdateLatestRecord: device_id=5, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.032 [event_task.cc:259] 619439 ConstructSqeForEventRecordTask: event_record: device_id=5, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.050 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.062 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.081 [task_submit.cc:573] 619439 AllocTaskAndSendStars: device_id=5, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.097 [stars_engine.cc:1122] 619439 SyncTask: Task Wait: device_id=5, stream_id=3, task_id=0, logicCqId=368. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.116 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.130 [stars_engine.cc:1071] 619439 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.148 [stars_engine.cc:1017] 619439 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.176 [stars_engine.cc:1027] 619439 ProcReport: taskType=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.190 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.208 [event.cc:249] 619439 UpdateLatestRecord: device_id=5, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.225 [event_task.cc:211] 619439 DoCompleteSuccessForEventRecordTask: event_record: device_id=5, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea57149c6c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.244 [event.cc:122] 619439 TryFreeEventIdAndCheckCanBeDelete: device_id=5, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.258 [task_res.cc:323] 619439 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=5, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.286 [npu_driver.cc:5618] 619439 GetSqEnable: deviceId=5, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.303 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.315 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.329 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.376 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.394 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.408 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.421 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.438 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.450 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.466 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.483 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.498 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.511 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.528 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.541 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.569 [stars_engine.cc:622] 619439 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=368, retCode=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.586 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:50.979.602 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.622 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:50.979.639 [memory.cpp:421]619439 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.52s ============================================================================================================== name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340880] [debug] [619756] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340887] [debug] [619756] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340893] [debug] [619756] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340900] [debug] [619756] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.359276] [debug] [619756] [lcal_comm.cpp:254] InitCommMem 5/8, localRank_ : 5, localRankSize_ : 8 success [2025-04-03 09:35:45.359422] [info] [619756] [lcal_comm.cpp:259] LcalCommInit 5/8 success and extraFlag:0 commArgs_.localRank : 5 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.390630] [error] [619791] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.395617] [error] [619791] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.398709] [error] [619791] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.988.511 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.988.551 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:50.988.629 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619439:281472858435616,MainProcess):2025-04-03-09:35:50.988.781 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.988.862 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.988.940 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.988.991 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:50.989.018 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.174 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.176 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.183 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.185 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.189 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.204.196 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.299 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.322 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.337 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.337 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.354 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.357 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.368 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.369 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.374 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.381 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.394 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.399 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.401 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.406 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.429 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.449 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.470 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.433 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.454 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.463 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.474 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.516 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.495 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.546 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.499 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.524 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.533 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.562 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.571 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.581 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.599 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.204.636 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.646 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.654 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.662 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.204.724 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.428.186 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.428.243 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.428.260 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.428.276 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.428.293 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.428.309 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.428.326 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.460.174 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.460.203 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.460.224 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.460.237 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.460.250 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.460.262 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.460.277 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.460.823 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.460.904 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619439,ffff21285fa0,python3.9):2025-04-03-09:35:51.460.979 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.063 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.084 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619439,ffff21a95fa0,python3.9):2025-04-03-09:35:51.461.126 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.200 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.219 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.269 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619439,ffff13ffefa0,python3.9):2025-04-03-09:35:51.461.335 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.601 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.631 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619439,ffff20a75fa0,python3.9):2025-04-03-09:35:51.461.670 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.762 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.783 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.804 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.461.826 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.461.939 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.461.962 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.461.981 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.001 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.018 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.035 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.052 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.067 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.082 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.098 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.118 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.131 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.145 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.161 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.172 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.193 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.213 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.225 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.240 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.252 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.265 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.290 [stream.cpp:131]619439 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.304 [api_impl.cc:1394] 619439 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.317 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.333 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.462.345 [api_error.cc:895] 619439 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.462.356 [stream.cpp:146]619439 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.405 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.472 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.538 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.557 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.572 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.749 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.462.771 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.522 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.549 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.568 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.584 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.603 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.828 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.850 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.866 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.932 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.952 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.463.972 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_5 [INFO] GE_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.475.741 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.475.785 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.475.811 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.475.828 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.849 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.475.864 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.475.880 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.892 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.475.904 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.475.928 [device.cpp:143]619439 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.948 [api_impl.cc:2999] 619439 SetDevice: device_id=5, deviceMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.964 [device.cc:306] 619439 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.976 [runtime.cc:3393] 619439 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.475.992 [runtime.cc:3560] 619439 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.476.007 [api_impl.cc:2987] 619439 NewDevice: SetCurRef = 0xaaab18d025e0,userDeviceId=5,deviceId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.476.022 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.476.043 [api_impl.cc:2992] 619439 NewDevice: New device success, user_device_id=5, die_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.476.060 [api_impl.cc:3025] 619439 SetDevice: SetDevice success, curCtx=0xaaab18d025e0, user_device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.476.075 [acl.cpp:115]619439 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.476.090 [acl.cpp:121]619439 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.476.105 [device.cpp:154]619439 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 5 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:51.476.130 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.532.730 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:51.532.757 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:51.532.776 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:51.532.792 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.532.812 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.532.900 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:51.556.904 [awatchdog_monitor.c:160](tid:619472) thread 619791 not exist [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.940.171 [trs_cb_event.c:119][ascend][curpid:619439,619548][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =832; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.940.213 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.940.230 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.940.249 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.940.262 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.940.275 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.940.289 [npu_driver.cc:3828] 619548 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=832. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.972.172 [trs_cb_event.c:119][ascend][curpid:619439,619549][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=5, tid =833; ret=16) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:51.972.199 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.972.216 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:51.972.230 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.972.250 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:51.972.265 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:51.972.279 [npu_driver.cc:3828] 619549 CqReportIrqWait: Irq wait start, deviceId=5, tsId=0, groupId=833. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.173 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.174 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.175 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.179 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.183 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.036.183 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.281 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.297 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.300 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.311 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.319 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.323 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.330 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.332 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.336 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.348 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.354 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.359 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.360 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.369 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.377 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.386 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.399 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.415 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.387 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.409 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.415 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.423 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.432 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.465 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.472 [tensor_data_transfer.cpp:1054]619553 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is _npu_log [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.488 [tensor_data_transfer.cpp:1056]619553 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.439 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.456 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.509 [api_impl.cc:6121] 619553 MemQueuePeek: Start to peek queue on devId 5, qid is 0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.522 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.533 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.541 [npu_driver.cc:4330] 619553 MemQueuePeek: peek mem queue, deviceId=5, qid=0, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.555 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.036.562 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.624 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.036.635 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.307 [op_base.cc:2136] [619439]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619439,fffe82fdefa0,python3.9):2025-04-03-09:35:52.341.322 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.341.337 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.356 [adapter_rts.cc:198] [619439][hrtGetDeviceRefresh]deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.371 [op_base.cc:2151] [619439][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.389 [op_base.cc:128] [619439][HcclGetDeviceId] deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.430 [op_base.cc:2168] [619439]com is not global com [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.452 [hcom_plugin.cc:381] [619439][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.487 [hccl_comm.cc:1375] [619439]UnRegistTaskAbortHandler begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.506 [task_abort_handler.cc:140] [619439]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.341.522 [api_impl.cc:4192] 619439 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.341.540 [runtime.cc:4698] 619439 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.564 [hccl_communicator.cc:2112] [619439][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.341.582 [hccl_communicator.cc:2114] [619439][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.598 [hccl_communicator.cc:2082] [619439]start to stop the backGround Thread [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.341.615 [heartbeat.cc:1203] [619439]Heartbeat has been destroyed [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.629 [heartbeat.cc:451] [619439][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.341.645 [heartbeat.cc:452] [619439]Heartbeat has been destroyed [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.676 [hccl_impl.cc:69] [619439]start hccl resources destruction:deviceLogicId[5] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.341.692 [heartbeat.cc:1203] [619439]Heartbeat has been destroyed [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.703 [heartbeat.cc:451] [619439][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.341.714 [heartbeat.cc:452] [619439]Heartbeat has been destroyed [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.341.810 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.341.929 [task_exception_handler.cc:1232] [619439]deInit taskFailCallback [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.343.214 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.343.236 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.343.273 [api_error.cc:1278] 619439 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.343.290 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.317 [hccl_socket_manager.cc:81] [619439][DeInit][Server]ip[192.168.2.194] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.335 [hccl_socket_manager.cc:83] [619439][DeInit][Server]ip[192.168.2.194] port[16666] [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.343.349 [hccl_socket.cc:232] [619439][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.360 [network_manager.cc:69] [619439]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.372 [network_manager.cc:458] [619439]NetworkManager devicePhyId_[5] vnic stopped. refcount[0] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:52.343.393 [ra_host.c:911]tid:619439,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[5], local_ip[192.168.2.194] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:52.343.732 [ra_host.c:404]tid:619439,ra_socket_deinit(404) : Input parameters: phy_id[5] family[2] local_ip[192.168.2.194] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.882 [hccl_socket.cc:68] [619439][HcclSocket][DeInit] device[5] stops listen on ip[192.168.2.194], port[16666] success, socketType[0]. [WARNING] HCCL(619439,python3.9):2025-04-03-09:35:52.343.904 [hccl_socket.cc:232] [619439][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.923 [hccl_communicator.cc:1741] [619439][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.343.940 [network_manager.cc:69] [619439]NetworkManager::GetInstance deviceLogicID[5]. [INFO] HCCP(619439,python3.9):2025-04-03-09:35:52.343.955 [ra_host.c:248]tid:619439,ra_deinit(248) : Input parameters: phy_id[5], nic_position:[1] [INFO] HCCP(619439,python3.9):2025-04-03-09:35:52.343.967 [ra_hdc.c:651]tid:619439,ra_hdc_deinit(651) : hdc deinit start! phy_id[5] [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.344.062 [hdc_client.c:417][ascend][curpid:619439,619439][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4276) [INFO] ROCE(619439,python3.9):2025-04-03-09:35:52.344.113 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619439,python3.9):2025-04-03-09:35:52.344.129 [ra_hdc.c:685]tid:619439,ra_hdc_deinit(685) : hdc deinit OK! phy_id[5] [INFO] ROCE(619439,python3.9):2025-04-03-09:35:52.344.142 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.184 [tsd_client.cpp:189][TsdCapabilityGet][tid:619439] TsdCapabilityGet Begin. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.211 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.224 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.236 [process_mode_manager.cpp:1042][CapabilityGet][tid:619439] [ProcessModeManager] enter into CapabilityGet process deviceId[5] type[6] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.344.250 [adapter_tdt.cc:66] [619439]Get TsdCapability success. deviceLogicId[5] [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.265 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.277 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.290 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619439] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.307 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619439] add close subproc:31150, proctype:0 [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.344.339 [hdc_client.cpp:154][TsdRecvData][tid:619439] recv timeout is:150000 ms [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.416.395 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619439] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.416.416 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619439] [TsdClient] DeviceMsgProc recvMsg realDeviceId[5] msgType[61] localDevId[5] rspCode[0]heterogeneousSubPid[31150], tsdSupportLevel_[63] [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.416.436 [hdc_client.cpp:172][TsdRecvData][tid:619439] [deviceId=5] the Tsd recv data already [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.416.450 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619439] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.416.466 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619439] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.480 [adapter_tdt.cc:52] [619439]Open TsdProcess success. deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.496 [network_manager.cc:405] [619439][DeInit]finish CloseHccpProcess, phy_id[5], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.511 [network_manager.cc:409] [619439]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.526 [hccl_communicator_attrs.cc:1307] [619439][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.542 [hccl_communicator_attrs.cc:1275] [619439]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.559 [notify_pool_impl.cc:43] [619439]NotifyPoolImpl Destroy. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.577 [notify_pool_impl.cc:56] [619439]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.600 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.615 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.628 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.642 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.654 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.677 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.689 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.704 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.723 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.737 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.748 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=0. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.416.765 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=0.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.779 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.789 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.801 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.811 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.825 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.838 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.848 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.861 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=1. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.416.875 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=1.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.890 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.901 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.914 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.922 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.932 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.943 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.416.957 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.968 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=2. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.416.982 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=2.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.416.996 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.006 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.018 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.028 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.053 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.064 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.078 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.089 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=3. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.417.102 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=3.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.114 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.127 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.138 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.149 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.163 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.175 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.186 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.200 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=4. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.417.216 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=4.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.229 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.243 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.255 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.265 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.280 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.292 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.303 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.313 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.325 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.339 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.352 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.368 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=6. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.417.383 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=6.) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.397 [api_impl.cc:3100] 619439 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.407 [api_impl.cc:3111] 619439 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.423 [api_impl.cc:3130] 619439 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.434 [p2p_mgmt.cc:171] [619439]local logic id:5, local physic id:5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.445 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.458 [api_impl.cc:3520] 619439 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.472 [p2p_mgmt.cc:178] [619439]disable p2p: local logic id:5, remote physic id:7. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.483 [api_impl.cc:3184] 619439 DisableP2P: Disable P2P devIdDes=5, phyIdSrc=7. [INFO] DRV(619439,python3.9):2025-04-03-09:35:52.417.497 [devdrv_pcie.c:890][ascend][curpid:619439,619439][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=5; peer_phy_devid=7.) [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.511 [hccl_communicator.cc:2043] [619439]Profiling is deinitiated. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.521 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.541 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.417.557 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.571 [opexecounter.cc:103] [619439]free counter mem resource [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.417.586 [atrace_api.c:118](tid:619439) AtraceDestroy start. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.417.601 [tracer_mgr_operate.c:248](tid:619439) destroy object HCCL_619756_5, exitSave(true). [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.417.616 [atrace_api.c:120](tid:619439) AtraceDestroy end. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.417.626 [adapter_trace.cc:33] [619439]Call TraceDestroy, Params:handle[0xaaab07266908] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.418.871 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.418.904 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.418.928 [queue_notify_manager.cc:100] [619439]QueueNotifyManager Destroy. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.418.945 [queue_notify_manager.cc:106] [619439]QueueNotifyManager Destroy success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.068 [queue_notify_manager.cc:100] [619439]QueueNotifyManager Destroy. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.085 [queue_notify_manager.cc:106] [619439]QueueNotifyManager Destroy success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.129 [ccl_buffer_manager.cc:125] [619439][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.148 [ccl_buffer_manager.cc:147] [619439][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.163 [ccl_buffer_manager.cc:160] [619439][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.179 [ccl_buffer_manager.cc:163] [619439][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.194 [ccl_buffer_manager.cc:166] [619439][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.226 [op_base.cc:128] [619439][HcclGetDeviceId] deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.247 [op_base.cc:2196] [619439]op_base comm destroy complete, take time [77911]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[5]. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.419.276 [device.cpp:183]619439 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.295 [api_impl.cc:3419] 619439 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.309 [runtime.cc:3444] 619439 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.326 [api_impl.cc:3449] 619439 DeviceReset: Succ, userDeviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.419.340 [device.cpp:189]619439 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 5. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.419.381 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.419.408 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.419.423 [hcom_executor.cc:40] [619439]Hcom Excutor Finalize start. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.543 [api_impl.cc:3419] 619567 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.575 [runtime.cc:3444] 619567 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.582 [api_impl.cc:3419] 619565 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.592 [api_impl.cc:3449] 619567 DeviceReset: Succ, userDeviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.611 [runtime.cc:3444] 619565 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.626 [api_impl.cc:3449] 619565 DeviceReset: Succ, userDeviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.800 [api_impl.cc:3419] 619568 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.830 [runtime.cc:3444] 619568 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.846 [api_impl.cc:3449] 619568 DeviceReset: Succ, userDeviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.922 [api_impl.cc:3419] 619566 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.949 [runtime.cc:3444] 619566 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.419.963 [api_impl.cc:3449] 619566 DeviceReset: Succ, userDeviceId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.110 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.420.133 [adapter_rts.cc:321] [619439][hrtCtxSetCurrent] success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.420.147 [stream.cc:182] [619439]Switch Ctx ret[0], curCtx[0xaaab18d025e0], setCtx[0xaaab18d025e0], stream id[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.190 [api_impl.cc:1228] 619439 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.420.215 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.420.264 [single_op_manager.cc:50]619439 ReleaseResource:ReleaseResource in. resource id = 0xaaab23267960 [INFO] GE(619439,python3.9):2025-04-03-09:35:52.420.288 [caching_allocator.cc:402]619439 TryFreeBlocks:Try free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.420.303 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:52.420.333 [npu_memory_allocator.cc:72]619439 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.420.361 [stream_executor.cpp:197]619439 RemoveExecutor: To remove executor by stream = 187652005853536 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.420.397 [acl_resource_manager.cpp:823]619439 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.411 [stream.cc:852] 619439 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.427 [stream.cc:938] 619439 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.438 [stream.cc:1507] 619439 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.461 [stream_sqcq_manage.cc:451] 619439 FreeLogicCq: Return(0), threadIdentifier(281472858435616), devId(5), tsId(0), cqId(370), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.487 [npu_driver.cc:6163] 619439 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=5, logicCq=370. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.420.525 [npu_driver.cc:1180] 619439 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=370, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.425.843 [stream.cc:237] 619439 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.425.870 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.425.889 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.426.532 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.426.553 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.426.570 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.426.590 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=5, device_id=5, tsId=0, streamFlag=1536 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.427.208 [stream.cc:185] [619439][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab23267960], stream id[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.230 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.427.247 [adapter_rts.cc:321] [619439][hrtCtxSetCurrent] success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.427.260 [stream.cc:193] [619439]Restore Ctx ret[0], setCtx[0xaaab18d025e0], stream id[5] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.306 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.427.327 [adapter_rts.cc:321] [619439][hrtCtxSetCurrent] success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.427.340 [stream.cc:182] [619439]Switch Ctx ret[0], curCtx[0xaaab18d025e0], setCtx[0xaaab18d025e0], stream id[4] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.356 [api_impl.cc:1228] 619439 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.427.373 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.427.390 [single_op_manager.cc:50]619439 ReleaseResource:ReleaseResource in. resource id = 0xaaab231eb900 [INFO] GE(619439,python3.9):2025-04-03-09:35:52.427.406 [caching_allocator.cc:402]619439 TryFreeBlocks:Try free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.427.420 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:52.427.449 [npu_memory_allocator.cc:72]619439 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.427.462 [stream_executor.cpp:197]619439 RemoveExecutor: To remove executor by stream = 187652005345536 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.427.475 [acl_resource_manager.cpp:823]619439 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.490 [stream.cc:852] 619439 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.508 [stream.cc:938] 619439 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.521 [stream.cc:1507] 619439 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.545 [stream_sqcq_manage.cc:451] 619439 FreeLogicCq: Return(0), threadIdentifier(281472858435616), devId(5), tsId(0), cqId(369), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.569 [npu_driver.cc:6163] 619439 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=4, logicCq=369. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.427.606 [npu_driver.cc:1180] 619439 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=369, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.228 [stream.cc:237] 619439 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.252 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.270 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.769 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.791 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.806 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.432.823 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=4, device_id=5, tsId=0, streamFlag=1536 [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.378 [stream.cc:185] [619439][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab231eb900], stream id[4] [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.433.400 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.417 [adapter_rts.cc:321] [619439][hrtCtxSetCurrent] success [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.433 [stream.cc:193] [619439]Restore Ctx ret[0], setCtx[0xaaab18d025e0], stream id[4] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.462 [hcom_executor.cc:45] [619439][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.433.496 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.433.517 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.530 [hcom_ops_kernel_info_store.cc:3423] [619439]finalize hccl kernel info store. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.558 [hcom_common.cc:85] [619439][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[5] [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.585 [plugin_manager.cc:56] [619439]hccl ops plugin finalize start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.603 [hcom_plugin.cc:90] [619439]Finalize start. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.680 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.697 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.710 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.723 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.753 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.767 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.782 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.795 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.823 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.835 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.844 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.857 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.885 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.900 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.910 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.924 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.936 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.949 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.962 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.975 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.986 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.433.997 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.009 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.021 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.049 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.061 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.077 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.091 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.119 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.135 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.146 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.159 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.197 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.213 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.225 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.240 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.273 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.285 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.297 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.310 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.342 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.358 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.371 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.383 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.414 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.429 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.439 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.450 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.482 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.500 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.511 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.522 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.588 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.604 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.615 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.627 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.661 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.674 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.685 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.697 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.727 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.741 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.750 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.763 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.773 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.786 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.804 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.817 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.830 [hcom_plugin.cc:119] [619439]Finalize: HcomDestroy success. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.843 [plugin_manager.cc:62] [619439]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.434.891 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.434.913 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.925 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.942 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.954 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.966 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.977 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.434.993 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.003 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.015 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.027 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.041 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.050 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.063 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.075 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.088 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.098 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.109 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.119 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.130 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.143 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.158 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.170 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.184 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.195 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.205 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.218 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.231 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.251 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.264 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.280 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.295 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.307 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.317 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.328 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.340 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.353 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.367 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.379 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.392 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.404 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.414 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.425 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.440 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.449 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.463 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.474 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.489 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.503 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.516 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.527 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.540 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.552 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.565 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.576 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.586 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.598 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.609 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.622 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.636 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.646 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.656 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.674 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.688 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.699 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.712 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.723 [hcom_common.cc:958] [619439][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.733 [hcom_common.cc:961] [619439]Entry-HcomDestroy:void skip [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.749 [hcom.cc:337] [619439][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:52.435.761 [hcom.cc:340] [619439]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.435.781 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.435.800 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.435.990 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.016 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.034 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.050 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.063 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.077 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.093 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61027123200 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.110 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.124 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61027122688, total count : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.139 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.165 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.185 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.201 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.225 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61027123200 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.239 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.257 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61027122688, total count : 1 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.272 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116400.00MB,"idle mem":116399.99MB,"used by event mem":0.00MB,"eager free mem":116399.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.298 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116400.00MB,"idle mem":116399.99MB,"used by event mem":0.00MB,"eager free mem":116399.99MB}. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.318 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.336 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.352 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.365 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.384 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.399 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.413 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.428 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.444 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.462 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.488 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.505 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.522 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.535 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.551 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe7c100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.567 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe7c101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.436.591 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58200M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.612 [callback.cpp:37]619439 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.629 [callback.cpp:44]619439 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.670 [context.cpp:72]619439 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.689 [api_impl.cc:3717] 619439 ContextSetCurrent: set context=aaab18d025e0, flag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.703 [context.cpp:80]619439 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.718 [stream.cpp:75]619439 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619439,python3.9):2025-04-03-09:35:52.436.738 [stream_manage_func_registry.cc:22]619439 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.749 [stream.cpp:80]619439 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.764 [api_impl.cc:1228] 619439 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.780 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.436.795 [single_op_manager.cc:50]619439 ReleaseResource:ReleaseResource in. resource id = 0xaaab18e2db80 [INFO] GE(619439,python3.9):2025-04-03-09:35:52.436.812 [caching_allocator.cc:402]619439 TryFreeBlocks:Try free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.436.826 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:52.436.838 [npu_memory_allocator.cc:72]619439 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.851 [stream_executor.cpp:197]619439 RemoveExecutor: To remove executor by stream = 187651833650048 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.436.886 [acl_resource_manager.cpp:823]619439 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.899 [stream.cc:852] 619439 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.914 [stream.cc:938] 619439 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.928 [event.cc:1036] 619439 WaitForBusy: device_id=5, event_id=0 finished map already empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.946 [npu_driver.cc:626] 619439 EventIdFree: event id free begin, device_id=5, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.967 [raw_device.cc:1158] 619439 FreeEventIdFromDrv: device_id=5, event_id=0 tear down. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.981 [event.cc:61] 619439 ~Event: device_id=5, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.436.997 [npu_driver.cc:3803] 619439 SqCqFree: deviceId=5, tsId=0, sqId=120, cqId=120 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.041 [subscribe.cc:244] 619439 Delete: delete callback subscribe: groupId=832, threadId=281470983466912, devId=5, tsId=0, streamId=2, sqId=120, cqId=120 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.437.045 [trs_sqcq.c:1491][ascend][curpid:619439,619548][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.056 [stream.cc:1507] 619439 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.074 [npu_driver.cc:3833] 619548 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=832, drvRetCode=16. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.089 [stream_sqcq_manage.cc:451] 619439 FreeLogicCq: Return(0), threadIdentifier(281472858435616), devId(5), tsId(0), cqId(367), isFastCq(0). [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.092 [api_impl.cc:5058] 619548 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.108 [npu_driver.cc:6163] 619439 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=2, logicCq=367. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.116 [callback.cpp:132]619548 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.131 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.138 [npu_driver.cc:1180] 619439 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=367, drvFlag=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.150 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.165 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.177 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.189 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.202 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.212 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.224 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.258 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.270 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.284 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.295 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.310 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.322 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.333 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.344 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.355 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.370 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.386 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.398 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.408 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.419 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.431 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.443 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.457 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.470 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.484 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.494 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.503 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.515 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.530 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.541 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.553 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.574 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.588 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.602 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.612 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.623 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.633 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.644 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.658 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.670 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.684 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.695 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.707 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.718 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.732 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.745 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.756 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.768 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.778 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.790 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.805 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.817 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.828 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.839 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.848 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.859 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.882 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.893 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.907 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.917 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.930 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.940 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.956 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.970 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.437.982 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.437.997 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.007 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.018 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.031 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.043 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.055 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.068 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.079 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.094 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.108 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.121 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.132 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.142 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.157 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.171 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.183 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.201 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.214 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.226 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.238 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.249 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.261 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.273 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.284 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.299 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.311 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.323 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.337 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.351 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.363 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.379 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.390 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.401 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.411 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.423 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.433 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.444 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.458 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.469 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.481 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.499 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.509 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.522 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.538 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.549 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.558 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.568 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.580 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.594 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.606 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.616 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.626 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.639 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.651 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.666 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.677 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.691 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.702 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.716 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.728 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.743 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.753 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.765 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.775 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.786 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.807 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.818 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.828 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.840 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.851 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.864 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.876 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.889 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.906 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.917 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.929 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.941 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.952 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.963 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.438.977 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.989 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.438.999 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.012 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.024 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.035 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.048 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.058 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.068 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.080 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.094 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.118 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.129 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.139 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.150 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.161 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.171 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.183 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.195 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.208 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.222 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.234 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.245 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.260 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.272 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.284 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.293 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.307 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.321 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.337 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.351 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.364 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.377 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.388 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.398 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.412 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.431 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.443 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.453 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.466 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.480 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.492 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.504 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.516 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.528 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.539 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.550 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.565 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.577 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.590 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.600 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.610 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.621 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.635 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.645 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.657 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.669 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.680 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.692 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.708 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.724 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.734 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.745 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.758 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.773 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.789 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.804 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.819 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.833 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.848 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.863 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.877 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.888 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.899 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.909 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.923 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.933 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.946 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.439.961 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.976 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.439.989 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.002 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.010 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.024 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.037 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.058 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.071 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.084 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.096 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.107 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.118 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.129 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.141 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.154 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.187 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.200 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.209 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.221 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.240 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.251 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.260 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.273 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.283 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.297 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.311 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.325 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.339 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.352 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.363 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.376 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.414 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.426 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.437 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.449 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.463 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.475 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.488 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.501 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.515 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.528 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.538 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.551 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.563 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.575 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.588 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.600 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.613 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.626 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.635 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.646 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.659 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.671 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.683 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.696 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.707 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.726 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.739 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.749 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.760 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.771 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.785 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.798 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.810 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.823 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.833 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.844 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.854 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.871 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.882 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.893 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.905 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.917 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.929 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.942 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.955 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.969 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.440.980 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.440.991 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.002 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.018 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.039 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.054 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.067 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.082 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.097 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.109 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.121 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.135 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.146 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.159 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.172 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.189 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.203 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.216 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.229 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.241 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.255 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.265 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.277 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.290 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.303 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.316 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.329 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.344 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.361 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.372 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.384 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.399 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.412 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.423 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.434 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.445 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.455 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.470 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.477 [stream.cc:237] 619439 FreeStreamId: Free stream_id=2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.480 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.493 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.498 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.507 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=5, tsId=0, drvFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.512 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.530 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.541 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.551 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.562 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.575 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.589 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.600 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.613 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.623 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.634 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.659 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.670 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.681 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.693 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.704 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.719 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.732 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.744 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.759 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.769 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.780 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.793 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.805 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.815 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.828 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.840 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.852 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.865 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.877 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.889 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.900 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.912 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.927 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.939 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.962 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.441.976 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.441.990 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.005 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.016 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.028 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.039 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.049 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.047 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=5, tsId=0, drvFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.059 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.070 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.077 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.085 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=2, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.090 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.101 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=2, device_id=5, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.102 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.121 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.135 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.148 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.160 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.171 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.183 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.195 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.207 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.222 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.239 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.251 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.262 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.278 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.291 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.302 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.312 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.323 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.336 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.347 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.357 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.370 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.380 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.390 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.402 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.416 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.429 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.441 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.451 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.463 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.475 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.486 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.497 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.509 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.534 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.546 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.558 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.568 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.578 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.591 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.603 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.613 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.625 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.636 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.651 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.664 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.677 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.690 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.702 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.716 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.731 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.744 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.759 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.775 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.787 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.798 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.812 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.829 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.839 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.860 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.876 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.890 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.903 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.914 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.925 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.936 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.950 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.961 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.442.972 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.984 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.442.995 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.010 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.024 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.036 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.047 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.057 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.071 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.084 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.097 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.109 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.120 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.131 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.146 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.159 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.178 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.190 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.204 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.218 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.233 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.245 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.255 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.267 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.278 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.291 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.301 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.312 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.322 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.335 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.346 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.356 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.368 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.380 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.392 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.407 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.421 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.432 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.443 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.456 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.470 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.492 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.502 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.511 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.522 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.534 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.547 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.557 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.569 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.581 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.591 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.605 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.616 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.626 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.637 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.647 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.659 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.671 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.683 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.695 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.706 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.722 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.737 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.750 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.762 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.772 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.796 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.807 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.817 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.815 [stream.cpp:87]619439 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.828 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.844 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.858 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.443.858 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab18e2db80. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.868 [callback.cpp:116]619548 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.881 [stream.cpp:75]619439 aclrtDestroyStream: start to execute aclrtDestroyStream [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.885 [subscribe.cc:308] 619548 GetGroupIdByThreadId: can not find subscribe info, threadId=281470983466912 [INFO] GE(619439,python3.9):2025-04-03-09:35:52.443.901 [stream_manage_func_registry.cc:22]619439 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.905 [api_impl.cc:5043] 619548 ProcessReport: get groupId fail, threadIdentifier=281470983466912, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.912 [stream.cpp:80]619439 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.916 [callback.cpp:130]619548 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.443.928 [api_impl.cc:1228] 619439 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.443.942 [acl_resource_manager.cpp:814]619439 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.443.957 [single_op_manager.cc:50]619439 ReleaseResource:ReleaseResource in. resource id = 0xaaab18f5e630 [INFO] GE(619439,python3.9):2025-04-03-09:35:52.443.973 [caching_allocator.cc:402]619439 TryFreeBlocks:Try free blocks. [INFO] DEVICE(619439,ffff11fbefa0,python3.9):2025-04-03-09:35:52.443.975 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] GE(619439,python3.9):2025-04-03-09:35:52.443.984 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:52.443.997 [npu_memory_allocator.cc:72]619439 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.008 [stream_executor.cpp:197]619439 RemoveExecutor: To remove executor by stream = 187651834897968 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.021 [acl_resource_manager.cpp:823]619439 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.036 [stream.cc:852] 619439 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.066 [stream.cc:938] 619439 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.082 [event.cc:1036] 619439 WaitForBusy: device_id=5, event_id=1 finished map already empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.096 [npu_driver.cc:626] 619439 EventIdFree: event id free begin, device_id=5, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.114 [raw_device.cc:1158] 619439 FreeEventIdFromDrv: device_id=5, event_id=1 tear down. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.127 [event.cc:61] 619439 ~Event: device_id=5, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.139 [npu_driver.cc:3803] 619439 SqCqFree: deviceId=5, tsId=0, sqId=121, cqId=121 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.183 [subscribe.cc:244] 619439 Delete: delete callback subscribe: groupId=833, threadId=281470975012768, devId=5, tsId=0, streamId=3, sqId=121, cqId=121 [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.444.187 [trs_sqcq.c:1491][ascend][curpid:619439,619549][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=5; type=1; ret=16) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.204 [stream.cc:1507] 619439 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.215 [npu_driver.cc:3833] 619549 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=5, tsId=0, groupId=833, drvRetCode=16. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.233 [api_impl.cc:5058] 619549 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.232 [stream_sqcq_manage.cc:451] 619439 FreeLogicCq: Return(0), threadIdentifier(281472858435616), devId(5), tsId(0), cqId(368), isFastCq(0). [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.244 [callback.cpp:132]619549 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.254 [npu_driver.cc:6163] 619439 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=3, logicCq=368. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.255 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.277 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.288 [npu_driver.cc:1180] 619439 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=368, drvFlag=0. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.293 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.309 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.321 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.334 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.348 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.362 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.374 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.390 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.418 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.434 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.446 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.458 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.472 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.486 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.498 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.508 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.522 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.535 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.546 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.557 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.567 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.580 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.592 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.603 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.612 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.623 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.635 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.647 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.659 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.672 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.684 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.694 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.705 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.727 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.739 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.752 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.764 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.778 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.789 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.799 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.810 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.821 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.833 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.845 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.856 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.868 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.882 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.892 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.902 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.912 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.925 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.936 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.946 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.961 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.444.975 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.987 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.444.999 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.014 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.033 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.045 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.057 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.067 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.078 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.089 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.101 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.115 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.127 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.141 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.151 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.162 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.173 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.186 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.200 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.213 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.228 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.244 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.255 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.269 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.283 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.294 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.309 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.319 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.335 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.349 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.363 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.374 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.383 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.395 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.408 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.419 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.432 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.444 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.455 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.465 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.479 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.490 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.500 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.514 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.527 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.537 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.547 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.560 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.573 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.587 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.599 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.613 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.627 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.651 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.662 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.672 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.684 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.696 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.707 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.721 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.732 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.744 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.753 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.766 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.777 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.787 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.796 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.808 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.820 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.834 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.849 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.863 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.873 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.885 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.898 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.911 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.921 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.933 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.949 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.965 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.445.976 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.445.990 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.003 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.013 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.026 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.038 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.050 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.064 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.075 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.086 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.098 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.110 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.123 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.134 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.147 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.158 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.171 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.186 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.200 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.214 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.224 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.239 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.252 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.282 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.295 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.310 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.320 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.330 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.342 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.353 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.364 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.376 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.386 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.396 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.408 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.420 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.431 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.441 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.452 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.463 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.475 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.486 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.496 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.507 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.517 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.529 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.541 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.560 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.571 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.584 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.594 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.604 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.614 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.626 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.640 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.654 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.666 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.678 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.688 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.699 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.711 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.722 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.734 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.743 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.754 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.764 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.774 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.785 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.798 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.810 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.821 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.834 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.852 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.863 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.874 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.885 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.885 [stream.cc:237] 619439 FreeStreamId: Free stream_id=3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.895 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.906 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.909 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.919 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=5, tsId=0, drvFlag=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.924 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.936 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.947 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.962 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.446.974 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.985 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.446.997 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.008 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.018 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.029 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.042 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.054 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.064 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.077 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.088 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.101 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.113 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.143 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.154 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.165 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.175 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.186 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.197 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.207 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.217 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.228 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.239 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.249 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.258 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.269 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.280 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.290 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.299 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.310 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.320 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.332 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.345 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.356 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.366 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.378 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.388 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.399 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=5, tsId=0, drvFlag=0 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.406 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.418 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.421 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.433 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.437 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.450 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=3, device_id=5, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.453 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.469 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.480 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.490 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.501 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.511 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.522 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.532 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.543 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.552 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.562 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.575 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.588 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.599 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.611 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.623 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.636 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.647 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.663 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.674 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.687 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.700 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.712 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.722 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.732 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.743 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.753 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.764 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.776 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.788 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.801 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.813 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.823 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.834 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.846 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.857 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.867 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.879 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.889 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.902 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.912 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.923 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.942 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.954 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.964 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.976 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.447.988 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.447.999 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.008 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.020 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.032 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.043 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.053 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.063 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.075 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.086 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.096 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.106 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.118 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.117 [event.cc:61] 619439 ~Event: device_id=5, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.130 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.141 [stream.cpp:87]619439 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.142 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.172 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.190 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab18f5e630. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.192 [callback.cpp:116]619549 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.209 [subscribe.cc:308] 619549 GetGroupIdByThreadId: can not find subscribe info, threadId=281470975012768 [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.207 [device.cpp:183]619439 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 5. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.225 [api_impl.cc:5043] 619549 ProcessReport: get groupId fail, threadIdentifier=281470975012768, reCode=0x7160004 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.236 [api_impl.cc:3419] 619439 DeviceReset: reset device, device_id=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.237 [callback.cpp:130]619549 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.248 [runtime.cc:3444] 619439 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.448.260 [api_impl.cc:3449] 619439 DeviceReset: Succ, userDeviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.271 [device.cpp:189]619439 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 5. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.291 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] DEVICE(619439,ffff117aefa0,python3.9):2025-04-03-09:35:52.448.305 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.327 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.352 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.369 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.381 [callback.cpp:37]619439 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.448.401 [callback.cpp:44]619439 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.422 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.448.445 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.512.721 [awatchdog_monitor.c:160](tid:619472) thread 619565 not exist [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.512.758 [awatchdog_monitor.c:160](tid:619472) thread 619566 not exist [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.512.779 [awatchdog_monitor.c:160](tid:619472) thread 619567 not exist [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:52.512.802 [awatchdog_monitor.c:160](tid:619472) thread 619568 not exist [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.748.726 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.748.760 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.748.791 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:52.748.805 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.748.823 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.748.896 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.178 [queue_client_comm.c:114][ascend][curpid:619439,619553][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.183 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.187 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.189 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.191 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:52.868.194 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.294 [npu_driver.cc:4338] 619553 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.312 [tensor_data_transfer.cpp:612]619553 acltdtReceiveTensorV2: queue is empty, device is 5, name is _npu_log [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.314 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.350 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.352 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.382 [tensor_data_transfer.cpp:1054]619555 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_dump [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.390 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.419 [tensor_data_transfer.cpp:1056]619555 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.394 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.422 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.427 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.456 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.470 [api_impl.cc:6121] 619555 MemQueuePeek: Start to peek queue on devId 5, qid is 1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.478 [tensor_data_transfer.cpp:1054]619561 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_scalar_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.485 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.490 [tensor_data_transfer.cpp:1054]619557 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_tensor_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.495 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.503 [npu_driver.cc:4330] 619555 MemQueuePeek: peek mem queue, deviceId=5, qid=1, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.529 [tensor_data_transfer.cpp:1056]619561 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.539 [tensor_data_transfer.cpp:1054]619559 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_image_summary [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.542 [tensor_data_transfer.cpp:1056]619557 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.547 [tensor_data_transfer.cpp:1054]619563 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 5, name is ms_histogram_summary [INFO] DEVICE(619439,fffedfc1cfa0,python3.9):2025-04-03-09:35:52.868.548 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.567 [api_impl.cc:6121] 619561 MemQueuePeek: Start to peek queue on devId 5, qid is 4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.572 [tensor_data_transfer.cpp:1056]619559 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.577 [api_impl.cc:6121] 619557 MemQueuePeek: Start to peek queue on devId 5, qid is 2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.579 [tensor_data_transfer.cpp:1056]619563 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.597 [npu_driver.cc:4330] 619561 MemQueuePeek: peek mem queue, deviceId=5, qid=4, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.605 [api_impl.cc:6121] 619559 MemQueuePeek: Start to peek queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.608 [npu_driver.cc:4330] 619557 MemQueuePeek: peek mem queue, deviceId=5, qid=2, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.613 [api_impl.cc:6121] 619563 MemQueuePeek: Start to peek queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.630 [npu_driver.cc:4330] 619559 MemQueuePeek: peek mem queue, deviceId=5, qid=3, timeout=800. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.636 [npu_driver.cc:4330] 619563 MemQueuePeek: peek mem queue, deviceId=5, qid=5, timeout=800. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.868.744 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is _npu_log [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.766 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:52.868.779 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=0. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:52.869.181 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is _npu_log [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.869.213 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:52.869.233 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:53.696.174 [queue_client_comm.c:114][ascend][curpid:619439,619561][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:53.696.188 [queue_client_comm.c:114][ascend][curpid:619439,619557][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:53.696.188 [queue_client_comm.c:114][ascend][curpid:619439,619559][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619439,python3.9):2025-04-03-09:35:53.696.206 [queue_client_comm.c:114][ascend][curpid:619439,619563][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.273 [npu_driver.cc:4338] 619561 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.696.287 [tensor_data_transfer.cpp:612]619561 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_scalar_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.324 [npu_driver.cc:4338] 619557 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.696.348 [tensor_data_transfer.cpp:612]619557 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.351 [npu_driver.cc:4338] 619559 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.696.368 [tensor_data_transfer.cpp:612]619559 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_image_summary [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.386 [npu_driver.cc:4338] 619563 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.696.411 [tensor_data_transfer.cpp:612]619563 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_histogram_summary [INFO] DEVICE(619439,fffec97aefa0,python3.9):2025-04-03-09:35:53.696.496 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619439,fffea7ffefa0,python3.9):2025-04-03-09:35:53.696.538 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619439,fffeca7cefa0,python3.9):2025-04-03-09:35:53.696.526 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619439,fffecb7eefa0,python3.9):2025-04-03-09:35:53.696.526 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.696.844 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is ms_histogram_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.885 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.696.901 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.697.341 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is ms_histogram_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.697.391 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.697.417 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.697.474 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is ms_scalar_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.697.489 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 4. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.697.500 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=4. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.697.860 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is ms_scalar_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.697.887 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.697.905 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.697.974 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is ms_image_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.697.991 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 3. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.698.000 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=3. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.698.432 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is ms_image_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.698.458 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.698.475 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.698.516 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is ms_tensor_summary [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.698.532 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.698.542 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=2. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.698.888 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is ms_tensor_summary [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.698.914 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.698.931 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [WARNING] DRV(619439,python3.9):2025-04-03-09:35:53.700.186 [queue_client_comm.c:114][ascend][curpid:619439,619555][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:53.700.268 [npu_driver.cc:4338] 619555 MemQueuePeek: queue empty [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.700.286 [tensor_data_transfer.cpp:612]619555 acltdtReceiveTensorV2: queue is empty, device is 5, name is ms_tensor_dump [INFO] DEVICE(619439,fffed8b87fa0,python3.9):2025-04-03-09:35:53.700.401 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.700.572 [tensor_data_transfer.cpp:973]619439 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 5, name is ms_tensor_dump [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.700.598 [api_impl.cc:6065] 619439 MemQueueDestroy: Start to destroy queue on devId 5, qid is 1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.700.609 [npu_driver.cc:3989] 619439 MemQueueDestroy: deviceId=5, qid=1. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.700.982 [tensor_data_transfer.cpp:977]619439 acltdtDestroyChannel: acltdtDestroyChannel success, device is 5, name is ms_tensor_dump [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.701.011 [device.cpp:183]619439 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.701.033 [api_impl.cc:3419] 619439 DeviceReset: reset device, device_id=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.701.048 [runtime.cc:3444] 619439 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.701.063 [api_impl.cc:3449] 619439 DeviceReset: Succ, userDeviceId=5. [INFO] ASCENDCL(619439,python3.9):2025-04-03-09:35:53.701.072 [device.cpp:189]619439 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 5. [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.104 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.136 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.183 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.200 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.222 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_5 [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.239 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.263 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.276 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.301 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.472 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.518 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.535 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.552 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.573 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.600 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.634 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.649 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(619439,fffecbffefa0,python3.9):2025-04-03-09:35:53.701.663 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] PS(619439,ffff12fdefa0,python3.9):2025-04-03-09:35:53.701.665 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] PS(619439,ffff127cefa0,python3.9):2025-04-03-09:35:53.701.676 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619439,fffea4f9efa0,python3.9):2025-04-03-09:35:53.701.717 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.818 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.837 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.852 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.873 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.887 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.900 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.701.914 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.043 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.069 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.120 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.136 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.150 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.168 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.181 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.213 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.227 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.241 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.266 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.310 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.325 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.660 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.679 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.711 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.727 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.742 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.702.757 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619439,ffff81bdb020,python3.9):2025-04-03-09:35:53.703.012 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:53.847.133 [awatchdog_monitor.c:160](tid:619472) thread 619569 not exist [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.964.887 [runtime.cc:5239] 619546 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:53.964.988 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:53.965.009 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619439,python3.9):2025-04-03-09:35:53.965.021 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619546] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.965.052 [runtime.cc:3077] 619546 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=5, devId=5, tdt status=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:53.965.199 [npu_driver.cc:551] 619546 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:54.146.539 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619439,ffff81bdb020,python3.9):2025-04-03-09:35:54.146.692 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.487.730 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.143 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.198 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.278 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.293 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.795 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.813 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.862 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.877 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.899 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.488.911 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.495 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.518 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.537 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.550 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.706 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.723 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.746 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.759 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.824 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.489.839 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.027 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.043 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.158 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.173 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.288 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.303 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.314 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.326 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.336 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.372 [heartbeat.cc:145] [619439][DeInit]heartbeat deinit begin [INFO] HCCL(619439,python3.9):2025-04-03-09:35:54.490.389 [heartbeat.cc:183] [619439][DeInit]heartbeat deinit end [INFO] DEVICE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:54.490.745 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 5, group name: hccl_world_group [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:54.490.831 [hash_data.cpp:63] >>> (tid:619439) HashData uninitialize success [INFO] PROFILING(619439,python3.9):2025-04-03-09:35:54.490.868 [msprof_reporter_mgr.cpp:270] >>> (tid:619439) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.493.107 [api_impl.cc:2003] 619439 DevFree: device free mem=0x12c041200000 [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:54.495.167 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619439,ffff81bdb020,python3.9):2025-04-03-09:35:54.495.200 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:54.495.365 [adx_dump_record.cpp:229][tid:619439] start to dump uninit [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:54.495.393 [adx_dump_record.cpp:239][tid:619439] dump uninit success [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:54.495.401 [adx_dump_record.cpp:674][tid:619551] exit record file thread [INFO] GE(619439,python3.9):2025-04-03-09:35:54.551.002 [operator_factory_impl.cc:392]619439 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.299 [operator_factory_impl.cc:392]619439 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.354 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.366 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.377 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.387 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.396 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.404 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.413 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.566.422 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.679.821 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.680.037 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] GE(619439,python3.9):2025-04-03-09:35:54.680.089 [opsproto_manager.cc:66]619439 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.737.438 [api_impl.cc:7151] 619439 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.737.692 [api_impl.cc:7135] 619439 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.739.527 [api_impl.cc:7151] 619439 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.739.694 [api_impl.cc:7135] 619439 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.741.139 [api_impl.cc:7121] 619439 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.744.859 [api_impl.cc:7121] 619439 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:54.752.523 [osal.cc:321] 619439 ThreadExit: ThreadExit start. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:55.176.773 [stars_engine.cc:1833] 619546 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:55.176.826 [stars_engine.cc:1836] 619546 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619439,ffff81bdb020,python3.9):2025-04-03-09:35:55.177.104 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619439,python3.9):2025-04-03-09:35:55.217.040 [ops_kernel_builder_registry.cc:71]619439 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619439,python3.9):2025-04-03-09:35:55.217.149 [ops_kernel_builder_registry.cc:41]619439 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:56.388.195 [adx_server_manager.cpp:236][tid:619552] server manager stop [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:56.560.857 [adx_hdc_epoll.cpp:57][tid:619439] start to close hdc epoll [INFO] GE(619439,python3.9):2025-04-03-09:35:56.670.757 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1da91490]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.672.716 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1da95ac0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.675.443 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1dabff40]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.678.953 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1da92ad0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.679.886 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1eecc0c0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.680.812 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1eef40b0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.681.698 [op_impl_registry_holder_manager.cc:31]619439 CloseHandle:start close handle, handle[0xaaab1efb7730]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.703.366 [mem_manager.cc:118]619439 Finalize:MemManager Finalize. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.703.411 [session_scope_mem_allocator.cc:73]619439 FreeAllMemory:Free all memory [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.710 [session_scope_mem_allocator.cc:73]619439 FreeAllMemory:Free all memory [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.744 [caching_allocator.cc:474]619439 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.775 [caching_allocator.cc:391]619439 FreeBlocks:Free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.789 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.798 [caching_allocator.cc:422]619439 FreeBlockBins:Free block bins. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.809 [caching_allocator.cc:474]619439 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.822 [caching_allocator.cc:391]619439 FreeBlocks:Free blocks. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.831 [caching_allocator.cc:358]619439 FreeCachedBlocks:Free cached blocks [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.840 [caching_allocator.cc:422]619439 FreeBlockBins:Free block bins. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.857 [mem_manager.cc:145]619439 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.873 [rts_caching_mem_allocator.cc:150]619439 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619439,python3.9):2025-04-03-09:35:56.705.940 [rts_caching_mem_allocator.cc:136]619439 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.567 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.622 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.634 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.645 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.654 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.664 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.702 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.715 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.724 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.734 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.745 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.754 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.762 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.792 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.802 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] HCCL(619439,python3.9):2025-04-03-09:35:56.828.810 [network_manager.cc:875] [619439]destroy all nic/vnic. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.835.956 [runtime.cc:1570] 619439 ~Runtime: deconstruct runtime [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.835.994 [runtime.cc:1577] 619439 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.263 [raw_device.cc:262] 619439 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.299 [engine.cc:188] 619439 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.326 [ringbuffer_maintain_task.cc:83] 619439 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.341 [stream.cc:1400] 619439 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.356 [stream.cc:2595] 619439 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.383 [npu_driver.cc:751] 619439 SqTaskSend: device_id=5, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.396 [stars_engine.cc:893] 619439 SendTask: Sq task send finished, device_id=5, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.412 [stream.cc:1494] 619439 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.430 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.445 [stream.cc:2886] 619439 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.459 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.470 [stream.cc:2886] 619439 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.482 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.495 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.509 [task.cc:185] 619439 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.523 [logger.cc:152] 619439 TaskFinished: device_id=5, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.544 [stars_engine.cc:437] 619439 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.558 [stream.cc:2886] 619439 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.572 [stars_engine.cc:524] 619439 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.584 [npu_driver.cc:5547] 619439 GetSqHead: deviceId=5, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.601 [stream.cc:852] 619439 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.617 [stream.cc:938] 619439 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.836.641 [stream.cc:1507] 619439 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.139 [stream_sqcq_manage.cc:451] 619439 FreeLogicCq: Return(0), threadIdentifier(281472858435616), devId(5), tsId(0), cqId(366), isFastCq(0). [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.199 [npu_driver.cc:6163] 619439 StreamUnBindLogicCq: success: deviceId=5, tsId=0, streamId=0, logicCq=366. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.253 [npu_driver.cc:1180] 619439 LogicCqFree: Succ:device_id=5,ts_id=0,type=2,logic_cq_id=366, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.269 [stream.cc:237] 619439 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.284 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.296 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.948 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.967 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.840.983 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.002 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=0, device_id=5, tsId=0, streamFlag=192 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.057 [stars_engine.cc:206] 619547 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.209 [stars_engine.cc:191] 619439 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.224 [osal.cc:225] 619439 Join: mmJoinTask failed, thread=281471220588448 does not exist! [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.235 [stars_engine.cc:150] 619439 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.251 [runtime.cc:5239] 619439 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=5, deviceId=5. [INFO] TDT(619439,python3.9):2025-04-03-09:35:56.841.268 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619439] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.283 [runtime.cc:3003] 619439 StopAicpuExecutor: TsdClose success, userDeviceId=5, devId=5, tdt status=0 [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:56.841.300 [atrace_api.c:118](tid:619439) AtraceDestroy start. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:56.841.319 [tracer_mgr_operate.c:248](tid:619439) destroy object RUNTIME_ATRACE_DEV5_TS0, exitSave(false). [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:56.841.334 [atrace_api.c:120](tid:619439) AtraceDestroy end. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.350 [stream.cc:237] 619439 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.363 [stream_sqcq_manage.cc:216] 619439 DeAllocStreamSqCq: deviceId=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.373 [npu_driver.cc:966] 619439 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.580 [npu_driver.cc:974] 619439 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=5, tsId=0, drvFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.595 [stream_sqcq_manage.cc:228] 619439 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.621 [npu_driver.cc:432] 619439 StreamIdFree: stream id free begin, device_id=5, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.841.636 [npu_driver.cc:441] 619439 StreamIdFree: Stream id free success, stream_id=1, device_id=5, tsId=0, streamFlag=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.853.595 [stars_engine.cc:58] 619439 ~StarsEngine: Destructor. [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.853.631 [engine.cc:84] 619439 ~Engine: Destructor. [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.474 [trs_dev_drv.c:370][ascend][curpid:619439,619439][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=5; fd=36; mmapFd=37) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.502 [devmm_svm.c:2238][ascend][curpid:619439,619439][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=5) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.515 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.534 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.549 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.566 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.577 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev5 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.593 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev5 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.610 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev5 Cached_size:1961984Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.620 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev5 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.635 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev5 Cached_size:2097152Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.646 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev5 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.661 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev5 Cached_size:2097152Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.674 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev5 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.700 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev5 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.716 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev5 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.732 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev5 Cached_size:23068672Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.743 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev5 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.759 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev5 Cached_size:6291456Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.770 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev5 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.784 [svm_mem_statistics.c:140][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev5 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.854.800 [svm_mem_statistics.c:146][ascend][curpid:619439,619439][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev5 Cached_size:16777216Bytes [INFO] DRV(619439,python3.9):2025-04-03-09:35:56.924.598 [drv_log_user.c:623][ascend][curpid:619439,619439][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=5; phyid=5; vfid=0; hostpid=619439; devpid=31089) [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.924.645 [npu_driver.cc:3045] 619439 DeviceClose: Close device success, device_id=5, tsId=0 [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:56.924.663 [raw_device.cc:125] 619439 ~Device: deconstruct device [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:57.033.303 [plugin_old_arch.cc:25] 619439 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:57.033.338 [runtime_keeper.cc:167] 619439 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619439,python3.9):2025-04-03-09:35:57.034.465 [task_fail_callback_data_manager.cc:63] 619439 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:57.147.784 [stacktrace_signal.c:188](tid:619439) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:57.148.716 [atrace_client_communication.c:141](tid:619439) send [end] msg. [INFO] DRV(619439,python3.9):2025-04-03-09:35:57.150.540 [hdc_core.c:1982][ascend][curpid:619439,619439][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4279; pid=619439) [INFO] IDEDD(619439,python3.9):2025-04-03-09:35:57.150.573 [hdc_api.cpp:671][tid:619439] connect succ, peer_node: 0, peer_devid: 5 [INFO] DRV(619439,python3.9):2025-04-03-09:35:57.150.644 [hdc_client.c:417][ascend][curpid:619439,619439][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4279) [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:57.150.703 [atrace_client_communication.c:164](tid:619439) send [end] msg success. [INFO] DRV(619439,python3.9):2025-04-03-09:35:58.218.255 [hdc_client.c:417][ascend][curpid:619439,619545][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4216) [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:58.218.321 [atrace_client_core.c:127](tid:619545) atrace receive thread exited, devId=5. [INFO] ATRACE(619439,python3.9):2025-04-03-09:35:58.218.877 [awatchdog_monitor.c:234](tid:619439) AwdMonitorExit [INFO] DRV(619439,python3.9):2025-04-03-09:35:58.392.903 [uda_user.c:147][ascend][curpid:619439,619439][drv][devmng][uda_char_dev_open]The fd and pid info: (fd=3; pid=619439) [INFO] DRV(619439,python3.9):2025-04-03-09:35:58.393.043 [uda_user.c:317][ascend][curpid:619439,619439][drv][devmng][_uda_init]User info. (admin_flag=1; local_flag=0; max_dev_num=100; max_udev_num=1124; uda_dev_num_davinci=8; uda_dev_num_kunpeng=0) output_allreduce/worker_6.log0000644000175100017500000665021014773362776017133 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619443,ffff964cc020,python3.9):2025-04-03-09:35:26.990.489 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.004.089 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 6 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.146 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.238 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.284 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.304 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.322 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.341 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.360 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.378 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.395 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.413 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.429 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.445 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.462 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.477 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.494 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.514 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.530 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.546 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.563 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.576 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.591 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.619 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.633 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.656 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.673 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.686 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.714 [ir_data_type_symbol_store.cc:240]619443 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.733 [ir_data_type_symbol_store.cc:240]619443 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.751 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.765 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.781 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.799 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.815 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.828 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.844 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.859 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.875 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.888 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.901 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.917 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.935 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.950 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.963 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.985 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.429.998 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.012 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.029 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.046 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.059 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.079 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.094 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.107 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.123 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.139 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.154 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.169 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.183 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.199 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.214 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.230 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.245 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.261 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.276 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.291 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.303 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.320 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.336 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.350 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.389 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.405 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.420 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.436 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.450 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.465 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.484 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.500 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.513 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.528 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.543 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.557 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.575 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.589 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.605 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.619 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.635 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.647 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.660 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.677 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.691 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.705 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.722 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.740 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.755 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.770 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.793 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.811 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.824 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.836 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.852 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.867 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.882 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.894 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.908 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.924 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.941 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.955 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.968 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.982 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.430.997 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.008 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.021 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.035 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.051 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.062 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.078 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.094 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.109 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.125 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.136 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.164 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.181 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.198 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.214 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.229 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.251 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.268 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.282 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.297 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.309 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.327 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.344 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.360 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.377 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.393 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.406 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.422 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.435 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.450 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.463 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.478 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.491 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.507 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.518 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.534 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.549 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.573 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.589 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.600 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.615 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.630 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.643 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.658 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.669 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.683 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.699 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.714 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.729 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.748 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.768 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.782 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.800 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.816 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.832 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.843 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.857 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.873 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.887 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.902 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.917 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.937 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.952 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.968 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.431.984 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.002 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.019 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.035 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.050 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.065 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.080 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.094 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.112 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.127 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.146 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.190 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.209 [ir_data_type_symbol_store.cc:173]619443 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.221 [ir_data_type_symbol_store.cc:219]619443 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619443,python3.9):2025-04-03-09:35:27.432.239 [ir_data_type_symbol_store.cc:182]619443 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.241 [awatchdog_core.c:99](tid:619443) watchdog get platform host side [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.398 [awatchdog_monitor.c:127](tid:619443) create monitor thread successfully, tid :281472566128544 [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.420 [awatchdog_monitor.c:226](tid:619443) AwdMonitorInit [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.613 [trace_attr.c:99](tid:619443) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.643 [trace_attr.c:189](tid:619443) attr init success, timeout=0ms. [WARNING] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.658 [trace_system_api.c:115](tid:619443) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.716 [trace_recorder.c:40](tid:619443) home_directory=/home/jenkins. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.735 [trace_recorder.c:121](tid:619443) use root path: /home/jenkins/ascend [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.747 [trace_event.c:151](tid:619443) init trace event success [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.439.792 [atrace_client_core.c:185](tid:619443) atrace client init success. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.118 [stacktrace_dumper.c:218](tid:619443) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.143 [trace_system_api.c:115](tid:619443) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.236 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 2 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.254 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 15 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.265 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 3 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.278 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 4 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.291 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 5 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.302 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 6 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.313 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 7 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.326 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 8 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.338 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 11 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.351 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 24 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.365 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 25 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.376 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 31 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.388 [stacktrace_signal.c:165](tid:619443) register signal handler for signo 35 succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.402 [stacktrace_signal.c:168](tid:619443) trace signal init succeed. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:27.441.412 [trace_core.c:51](tid:619443) trace init success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.442.116 [task_fail_callback_data_manager.cc:58] 619443 TaskFailCallBackManager: Constructor. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.452.388 [operator_factory_impl.cc:232]619443 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619443,python3.9):2025-04-03-09:35:27.452.424 [operator_factory_impl.cc:246]619443 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619443,python3.9):2025-04-03-09:35:27.452.439 [operator_factory_impl.cc:239]619443 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619443,python3.9):2025-04-03-09:35:27.459.966 [adapter_rts.cc:2720] [619443][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.474.896 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.474.975 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:27.475.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:27.485.170 [prof_cann_plugin.cpp:256] (tid:619443) Register module[6] callback. [INFO] OP(619443,python3.9):2025-04-03-09:35:27.536.176 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619443] cachelimit is 10000 [INFO] OP(619443,python3.9):2025-04-03-09:35:27.540.261 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619443] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619443,python3.9):2025-04-03-09:35:27.540.316 [op_dfx.cpp:54][NNOP][SystemConfig][619443] enable debug kernel flag 0. [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:27.540.338 [prof_cann_plugin.cpp:256] (tid:619443) Register module[61] callback. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.594.249 [mem_manager.cc:138]619443 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.601.411 [cache_policy.cc:47]619443 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.293 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.423 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.528 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.558 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.582 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.604 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.626 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.647 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.671 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.693 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.714 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.734 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.755 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.776 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.802 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.825 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.846 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.870 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.891 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.616.912 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.012 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.125 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.150 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.263 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.287 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.312 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.333 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.354 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.380 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.403 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.424 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.445 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.466 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.487 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.508 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.531 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.556 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.577 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.598 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.624 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.645 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.667 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.765 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.865 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.617.964 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.064 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.178 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.205 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.302 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.405 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.436 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.530 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.638 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.664 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.687 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.708 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.731 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.753 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.774 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.795 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.816 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.839 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.861 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.886 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.908 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.618.929 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.033 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.143 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.244 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.344 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.370 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.393 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.416 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.513 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.611 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.637 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.751 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.778 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.870 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.619.970 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.074 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.192 [addn.cc:29]619443 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.547 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.654 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.750 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.855 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619443,python3.9):2025-04-03-09:35:27.620.946 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.036 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.128 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.227 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.244 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.342 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.436 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619443,python3.9):2025-04-03-09:35:27.621.529 [symbolic_kernel_factory.cc:27]619443 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619443,python3.9):2025-04-03-09:35:27.628.270 [process_node_engine_manager.cc:140]619443 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619443,python3.9):2025-04-03-09:35:27.628.402 [process_node_engine_manager.cc:140]619443 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619443,python3.9):2025-04-03-09:35:27.634.982 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619443,python3.9):2025-04-03-09:35:27.635.018 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619443,python3.9):2025-04-03-09:35:27.635.446 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619443] start packet count timer [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:27.656.676 [prof_atls_plugin.cpp:117] (tid:619443) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:27.656.716 [prof_atls_plugin.cpp:117] (tid:619443) RegisterProfileCallback, callback type is 8 [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.734.763 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.735.247 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.787.774 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.787.902 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 6 [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.787.942 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.788.766 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.448 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.528 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.565 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.662 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.696 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.755 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.790.784 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.792.560 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.792.595 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.792.631 [acl.cpp:753]619443 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.792.707 [runtime_keeper.cc:78] 619443 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.614 [plugin_old_arch.cc:16] 619443 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaaf8724ba0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.647 [runtime_keeper.cc:149] 619443 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.659 [runtime_keeper.cc:221] 619443 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.708 [runtime.cc:1387] 619443 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.738 [runtime.cc:1163] 619443 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.764 [npu_driver.cc:4062] 619443 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.786 [runtime.cc:5096] 619443 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.802 [runtime.cc:1173] 619443 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.816 [runtime.cc:1183] 619443 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.835 [runtime.cc:1193] 619443 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.850 [profiler.cc:84] 619443 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.872 [runtime.cc:1209] 619443 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.897 [runtime.cc:1219] 619443 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.910 [runtime.cc:1230] 619443 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.923 [runtime.cc:1242] 619443 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.939 [runtime.cc:1264] 619443 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.796.950 [runtime.cc:1270] 619443 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.062 [runtime.cc:109] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.458 [runtime.cc:121] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.531 [runtime.cc:1314] 619443 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.548 [runtime.cc:1322] 619443 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.666 [runtime.cc:1338] 619443 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.727 [utils.cc:105] 619443 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.777 [runtime.cc:1664] 619443 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.814 [runtime.cc:1688] 619443 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.836 [runtime.cc:1953] 619443 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.864 [runtime.cc:1477] 619443 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:27.797.885 [prof_atls_plugin.cpp:210] (tid:619443) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.906 [heterogenous.cc:19] 619443 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:27.797.938 [api_c_soc.cc:70] 619443 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.797.955 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.797.969 [acl.cpp:760]619443 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.798.007 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.836.273 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.870.228 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.870.678 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.871.122 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.871.933 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.872.589 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.874.619 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.874.662 [acl.cpp:753]619443 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.874.688 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:27.874.702 [acl.cpp:760]619443 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:27.874.726 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.854.547 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.856.354 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.879.128 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.888.031 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.891.010 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.894.084 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.926.772 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619443,python3.9):2025-04-03-09:35:28.942.095 [platform_info.cpp:853]619443 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.873.333 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.875.178 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.875.314 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.877.586 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.877.727 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.877.845 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.877.974 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.878.085 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.879.240 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.879.381 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.318 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.454 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.572 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.686 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.792 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619443,python3.9):2025-04-03-09:35:29.881.901 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619443,python3.9):2025-04-03-09:35:30.486.507 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619443,python3.9):2025-04-03-09:35:34.560.836 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.821.471 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.821.564 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.821.604 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619443,python3.9):2025-04-03-09:35:34.866.374 [ops_kernel_builder_registry.cc:55]619443 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619443,python3.9):2025-04-03-09:35:34.866.482 [ops_kernel_builder_registry.cc:28]619443 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.904.086 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.904.208 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.904.300 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.904.325 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:34.904.341 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619443,python3.9):2025-04-03-09:35:35.738.419 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:37.482.965 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:37.483.630 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:37.494.900 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.584.955 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:37.585.039 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.332 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.382 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.404 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.437 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.640 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471591411616 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.748 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471582957472 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.773 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.586.891 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471574503328 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.587.002 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471566049184 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.587.034 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.587.055 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.587.255 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58720, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:37.587.296 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [WARNING] DISTRIBUTED(619443,ffff35366fa0,python3.9):2025-04-03-09:35:37.587.309 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58720 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.399 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.427 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.445 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.463 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.566 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58750, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.087.587 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619443,ffff36386fa0,python3.9):2025-04-03-09:35:38.087.603 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58750 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.587.663 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.587.684 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.587.914 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 6 has been registered successfully. [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:38.588.111 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619443,ffff27ffefa0,python3.9):2025-04-03-09:35:38.588.168 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619443,ffff27ffefa0,python3.9):2025-04-03-09:35:38.588.195 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:39.088.177 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:39.588.267 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.088.344 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(4/1200). [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.446 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3002.05 msec. [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.470 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.489 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.508 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 6 rank id: 6 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.526 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.569 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 9654 to 9909 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.594 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.101 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.623 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.648 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3002.38 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.674 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.828 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.849 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.879 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.909 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.928 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.588.947 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.590.696 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.590.718 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 9654 to 9909 [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.590.891 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:c4ea43c8-7efa-4497-8d7f-6ae496ed85d0, the ip:127.0.0.1, the port:9654 [INFO] PS(619443,ffff277eefa0,python3.9):2025-04-03-09:35:40.590.909 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619443,ffff277eefa0,python3.9):2025-04-03-09:35:40.590.939 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.592.558 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.592.713 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_6 has been registered successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.592.739 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:40.592.862 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.492 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 6 has been started successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.524 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.567 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.590 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.633 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 6, global rank size: 8 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.662 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3004.96 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.683 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 6 is devserver-c233-2 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.788 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.859 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.884 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 6 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.903 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 6 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.929 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [6]. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.593.964 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.277 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.594.036 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.594.076 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.594.102 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.594.135 [device.cpp:365]619443 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.594.270 [api_impl.cc:249] 619443 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.594.291 [api_impl.cc:6795] 619443 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.594.305 [device.cpp:371]619443 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.594.384 [callback.cpp:37]619443 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.594.424 [callback.cpp:44]619443 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.594.452 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.594.479 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.594.515 [device.cpp:143]619443 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.594.607 [npu_driver.cc:124] 619443 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.594.643 [api_impl.cc:2999] 619443 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.594.681 [runtime.cc:3602] 619443 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.594.869 [hdc_core.c:1982][ascend][curpid:619443,619443][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4233; pid=619443) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.595.001 [runtime.cc:3609] 619443 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.595.048 [atrace_client_communication.c:103](tid:619443) send [hello] msg. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.595.066 [atrace_client_communication.c:63](tid:619443) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.595.177 [hdc_core.c:1982][ascend][curpid:619443,619443][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4234; pid=619443) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:43.595.196 [hdc_api.cpp:671][tid:619443] connect succ, peer_node: 0, peer_devid: 6 [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.596.524 [hdc_client.c:417][ascend][curpid:619443,619443][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4234) [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.596.556 [atrace_client_communication.c:65](tid:619443) send msg by adx[finish], ret=0. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.596.574 [atrace_client_communication.c:97](tid:619443) msgType=1, magic=49959, version=1. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.596.598 [atrace_client_communication.c:135](tid:619443) send [hello] msg success. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.596.678 [atrace_client_core.c:160](tid:619443) start to receive device[6] trace log. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.596.709 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.596.726 [runtime.cc:2855] 619443 startAicpuExecutor: TsdOpen devId=6, device mode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.596.739 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.782 [client_manager.cpp:160][GetPlatformInfo][tid:619443] [TsdClient] mode[1] chipType[5] [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.596.810 [hdc_core.c:1982][ascend][curpid:619443,619577][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4236; pid=619443) [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.596.820 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619443] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:43.596.841 [hdc_api.cpp:671][tid:619577] connect succ, peer_node: 0, peer_devid: 6 [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.847 [client_manager.cpp:582][GetVisibleDevices][tid:619443] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.864 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.879 [client_manager.cpp:125][GetInstance][tid:619443] [ClientManager] GetInstance, deviceId[6], g_runningMode[0], begin saving instance [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.909 [client_manager.cpp:435][IsHostEnvironment][tid:619443] [TsdClient] logicDeviceId is [6], hostaicpunum[0] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.927 [client_manager.cpp:464][GetClientRunMode][tid:619443] runningMode:0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.938 [client_manager.cpp:127][GetInstance][tid:619443] [ClientManager] Current mode:1 [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.596.952 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619443] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.596.964 [client_manager.cpp:347][GetProfilingMode][tid:619443] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.596.998 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619443] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.014 [process_mode_base.cpp:149][GetLogLevel][tid:619443] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.597.080 [atrace_client_communication.c:177](tid:619577) send [start] msg. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.104 [process_mode_base.cpp:161][GetLogLevel][tid:619443] [TsdClient] Get Log Level[001] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.123 [process_mode_manager.cpp:71][OpenProcess][tid:619443] [ProcessModeManager] enter into open process deviceId[6] rankSize[0] [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.597.131 [atrace_client_communication.c:189](tid:619577) send [start] msg success. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.140 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619443] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.197 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[.], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.270 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[scene.info], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.343 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.427 [client_manager.cpp:264][CheckPackageExists][tid:619443] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.445 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[ascend_install.info], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.505 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[version.info], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.567 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[..], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.627 [client_manager.cpp:261][CheckPackageExists][tid:619443] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.757 [client_manager.cpp:268][CheckPackageExists][tid:619443] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.776 [client_manager.cpp:283][CheckPackageExists][tid:619443] [TsdClient][deviceId=6] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.788 [client_manager.cpp:286][CheckPackageExists][tid:619443] [TsdClient][deviceId=6] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.597.847 [devdrv_hccl.c:103][ascend][curpid:619443,619443][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619443) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.865 [process_mode_manager.cpp:396][InitTsdClient][tid:619443] [TsdClient][deviceId=6] get process sign success, procPid[619443] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.879 [process_mode_manager.cpp:403][InitTsdClient][tid:619443] [TsdClient] deviceId[6] begin to init hdc client [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.917 [hdc_common.cpp:67][InitMsgSize][tid:619443] HdcCommon::InitMsgSize Start [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.932 [hdc_common.cpp:80][InitMsgSize][tid:619443] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.597.948 [hdc_client.cpp:184][CreateHdcSession][tid:619443] HdcClient::CreateHdcSession Start [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.028 [hdc_client.cpp:208][CreateHdcSession][tid:619443] [HdcClient] deviceId: 6 connect session [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.598.060 [hdc_core.c:1982][ascend][curpid:619443,619443][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4237; pid=619443) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.078 [hdc_client.cpp:219][CreateHdcSession][tid:619443] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.092 [hdc_client.cpp:230][CreateHdcSession][tid:619443] [HdcClient] deviceId: 6 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.108 [version_verify.cpp:36][SetVersionInfo][tid:619443] VersionVerify: send client version to server [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.148 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.172 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.195 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.598.226 [internal_api.cpp:101][GetScheduleEnv][tid:619443] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.602 [version_verify.cpp:68][PeerVersionCheck][tid:619443] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.623 [version_verify.cpp:89][ParseVersionInfo][tid:619443] VersionVerify: pass client version info success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.640 [hdc_client.cpp:275][CheckHdcConnection][tid:619443] Service[2] create hdc success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.655 [version_verify.cpp:122][SpecialFeatureCheck][tid:619443] VersionVerify: new type[35], supported [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.598.672 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619443] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.702 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619443] [TsdClient][deviceId=6] [sessionId=1] wait package info respond [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.598.719 [hdc_client.cpp:154][TsdRecvData][tid:619443] recv timeout is:150000 ms [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.706 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619443] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.723 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619443] [PackageInfoMsgProc] [sessionID=1] deviceId[6], checkCode[56684994] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.737 [hdc_client.cpp:172][TsdRecvData][tid:619443] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.751 [version_verify.cpp:122][SpecialFeatureCheck][tid:619443] VersionVerify: new type[64], supported [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.765 [hdc_client.cpp:411][Destroy][tid:619443] enter HdcClient::Destroy() function [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.599.782 [hdc_client.c:417][ascend][curpid:619443,619443][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4237) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.806 [hdc_client.cpp:394][DestroyClient][tid:619443] begin drvHdcClientDestroy [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.823 [hdc_client.cpp:399][DestroyClient][tid:619443] end drvHdcClientDestroy [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.835 [hdc_client.cpp:377][ClearClientPtr][tid:619443] begin HdcClient::ClearClientPtr [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.846 [hdc_client.cpp:385][ClearClientPtr][tid:619443] end HdcClient::ClearClientPtr [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.861 [hdc_client.cpp:418][Destroy][tid:619443] end HdcClient::Destroy() function [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.887 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619443] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.906 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619443] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.599.926 [devdrv_hccl.c:103][ascend][curpid:619443,619443][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619443) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.942 [process_mode_manager.cpp:396][InitTsdClient][tid:619443] [TsdClient][deviceId=6] get process sign success, procPid[619443] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.954 [process_mode_manager.cpp:403][InitTsdClient][tid:619443] [TsdClient] deviceId[6] begin to init hdc client [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.970 [hdc_common.cpp:67][InitMsgSize][tid:619443] HdcCommon::InitMsgSize Start [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.599.991 [hdc_common.cpp:80][InitMsgSize][tid:619443] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.004 [hdc_client.cpp:184][CreateHdcSession][tid:619443] HdcClient::CreateHdcSession Start [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.057 [hdc_client.cpp:208][CreateHdcSession][tid:619443] [HdcClient] deviceId: 6 connect session [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.600.086 [hdc_core.c:1982][ascend][curpid:619443,619443][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4239; pid=619443) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.104 [hdc_client.cpp:219][CreateHdcSession][tid:619443] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.116 [hdc_client.cpp:230][CreateHdcSession][tid:619443] [HdcClient] deviceId: 6 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.130 [version_verify.cpp:36][SetVersionInfo][tid:619443] VersionVerify: send client version to server [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.146 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.181 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.199 [version_verify.cpp:52][SetVersionInfo][tid:619443] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.441 [version_verify.cpp:68][PeerVersionCheck][tid:619443] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.460 [version_verify.cpp:89][ParseVersionInfo][tid:619443] VersionVerify: pass client version info success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.471 [hdc_client.cpp:275][CheckHdcConnection][tid:619443] Service[2] create hdc success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.487 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619443] [TsdClient] tsd get process sign successfully, procpid[619443] signSize[48] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.505 [version_verify.cpp:114][SpecialFeatureCheck][tid:619443] VersionVerify: previous type[6], supported [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.533 [process_mode_manager.cpp:88][OpenProcess][tid:619443] [ProcessModeManager] deviceId[6] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619443,python3.9):2025-04-03-09:35:43.600.550 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619443] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.600.562 [hdc_client.cpp:154][TsdRecvData][tid:619443] recv timeout is:150000 ms [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.800.503 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619443] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.800.525 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619443] [TsdClient] DeviceMsgProc recvMsg realDeviceId[6] msgType[8] localDevId[6] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.800.538 [hdc_client.cpp:172][TsdRecvData][tid:619443] [deviceId=6] the Tsd recv data already [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.800.565 [devmm_svm.c:132][ascend][curpid:619443,619443][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=6) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.802.724 [devmm_virt_interface.c:450][ascend][curpid:619443,619443][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.803.057 [drv_log_user.c:623][ascend][curpid:619443,619443][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=6; devid=6; vfid=0; hostpid=619443; devpid=30773) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.803.092 [devmm_svm.c:122][ascend][curpid:619443,619443][drv][devmm][devmm_set_device_info]Device info. (devid=6; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.803.353 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619443] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.803.374 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619443] enter into OpenInHost deviceid[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.803.401 [client_manager.cpp:435][IsHostEnvironment][tid:619443] [TsdClient] logicDeviceId is [6], hostaicpunum[0] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.803.418 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619443] host cpu not support [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.803.428 [process_mode_manager.cpp:121][OpenProcess][tid:619443] [TsdClient][deviceId=6] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.803.444 [runtime.cc:2879] 619443 startAicpuExecutor: userDeviceId=6, devId=6, tsdOpen ret=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.803.459 [runtime.cc:2891] 619443 startAicpuExecutor: TsdOpen success, devId=6, ret=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.803.479 [runtime.cc:3627] 619443 DeviceRetain: new dev ok id=6, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.803.496 [raw_device.cc:936] 619443 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.803.509 [npu_driver.cc:2968] 619443 DeviceOpen: Open device start, device_id=6, tsId=0. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.803.598 [trs_sqcq.c:458][ascend][curpid:619443,619443][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=6) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.804.093 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=6; num=2044) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.804.580 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=7; num=2044) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.804.835 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=10; num=1024) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.088 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=11; num=1024) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.578 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=12; num=2048) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.606 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=8; num=16) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.628 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=9; num=16) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.648 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=13; num=4) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.805.668 [trs_sqcq.c:394][ascend][curpid:619443,619443][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=6; tsId=0; id_type=14; num=10) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.807.955 [trs_dev_drv.c:302][ascend][curpid:619443,619443][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=6; fd=36; mmapFd=37) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.807.986 [devmm_svm.c:132][ascend][curpid:619443,619443][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=6) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.808.006 [devmm_svm.c:122][ascend][curpid:619443,619443][drv][devmm][devmm_set_device_info]Device info. (devid=6; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.808.073 [npu_driver.cc:3011] 619443 DeviceOpen: Open device success, device_id=6, tsId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.808.092 [raw_device.cc:292] 619443 Init: chipType:0x5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.808.129 [raw_device.cc:304] 619443 Init: isAddrFlat:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.808.149 [raw_device.cc:320] 619443 Init: GetDevInfo, device_id=6, hardwareVersion=0x503 [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.808.175 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.808.190 [npu_driver.cc:1578] 619443 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.809.132 [raw_device.cc:352] 619443 Init: Get phy info, phy chip id 6, phy die id 0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.809.154 [runtime.cc:2771] 619443 GetTsdQos: [TSD] start get qos, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.809.168 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.182 [tsd_client.cpp:189][TsdCapabilityGet][tid:619443] TsdCapabilityGet Begin. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.197 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.210 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.224 [process_mode_manager.cpp:1042][CapabilityGet][tid:619443] [ProcessModeManager] enter into CapabilityGet process deviceId[6] type[0] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.240 [process_mode_manager.cpp:386][InitTsdClient][tid:619443] [TsdClient] tsd client has been inited before [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.255 [version_verify.cpp:114][SpecialFeatureCheck][tid:619443] VersionVerify: previous type[38], supported [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.290 [process_mode_manager.cpp:1065][CapabilityGet][tid:619443] [TsdClient][deviceId=6] [sessionId=1] [type=0]send capability success [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.809.308 [hdc_client.cpp:154][TsdRecvData][tid:619443] recv timeout is:150000 ms [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.810.257 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619443] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.810.276 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619443] [TsdClient] PidQosMsgProc recvMsg realDeviceId[6] msgType[39] localDevId[6] rspCode[0],pidqos[1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.810.290 [hdc_client.cpp:172][TsdRecvData][tid:619443] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.810.303 [process_mode_manager.cpp:1070][CapabilityGet][tid:619443] [TsdClient][logicDeviceId_=6][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.319 [runtime.cc:2782] 619443 GetTsdQos: [TSD] get qos success, userDeviceId=6, devId=6, qos=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.342 [engine.cc:78] 619443 Engine: Constructor. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.355 [stars_engine.cc:51] 619443 StarsEngine: Constructor. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.369 [engine_factory.cc:26] 619443 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.381 [engine_factory.cc:38] 619443 CreateEngine: Engine init success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.393 [raw_device.cc:364] 619443 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.409 [spm_pool.cc:42] 619443 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.532 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.551 [spm_pool.cc:55] 619443 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.563 [raw_device.cc:380] 619443 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.810.579 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.596 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.810.608 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.810.625 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.816.761 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.816.792 [task_allocator.cc:46] 619443 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.816.826 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.816.842 [h2d_copy_mgr.cc:45] 619443 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.816.857 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.816.874 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.817.804 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.817.828 [task_allocator.cc:46] 619443 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.817.848 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.817.864 [h2d_copy_mgr.cc:45] 619443 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.817.878 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.817.896 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.679 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.701 [task_allocator.cc:46] 619443 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.731 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.748 [h2d_copy_mgr.cc:45] 619443 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.763 [uma_arg_loader.cc:86] 619443 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.781 [task_allocator.cc:46] 619443 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.805 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.818.820 [uma_arg_loader.cc:93] 619443 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.601 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.622 [uma_arg_loader.cc:102] 619443 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.635 [uma_arg_loader.cc:104] 619443 Init: ALLOC PCIE is support[1] [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.819.649 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.666 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.811 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.830 [h2d_copy_mgr.cc:45] 619443 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.845 [raw_device.cc:393] 619443 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.859 [raw_device.cc:398] 619443 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.819.875 [task_allocator.cc:46] 619443 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.041 [buffer_allocator.cc:61] 619443 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.260 [raw_device.cc:109] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.676 [raw_device.cc:121] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.696 [raw_device.cc:425] 619443 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.756 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.843 [stream.cc:2246] 619443 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.883 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.820.901 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.541 [npu_driver.cc:6125] 619443 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.639 [npu_driver.cc:6137] 619443 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=305, streamId=0, logicCq=305, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.674 [npu_driver.cc:6189] 619443 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=0, logicCq=305. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.709 [npu_driver.cc:5916] 619443 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f41fd000, addr=dffff41fd000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.726 [stream.cc:653] 619443 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.808 [stream.cc:276] 619443 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.826 [stream.cc:688] 619443 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=6, streamResId=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.845 [stream_state_callback_manager.cc:45] 619443 Notify: not reg stream state callback func! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.861 [raw_device.cc:593] 619443 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.889 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.915 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=16 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.821.931 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.032 [tsch_stream.cc:45] 619443 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=6 streamResId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.051 [raw_device.cc:1219] 619443 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.132 [osal.cc:309] 619443 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.282 [osal.cc:182] 619443 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.308 [osal.cc:309] 619443 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.387 [stars_engine.cc:1785] 619590 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.389 [osal.cc:182] 619443 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.407 [stars_engine.cc:201] 619591 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.430 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.822.447 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.822.464 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.822.480 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.504 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.562 [npu_driver.cc:6306] 619590 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.590 [stars_engine.cc:1855] 619590 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.606 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.772 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.810 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.834 [task_info.cc:1596] 619443 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.853 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.867 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.891 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.908 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.924 [stars_engine.cc:1122] 619443 SyncTask: Task Wait: device_id=6, stream_id=0, task_id=0, logicCqId=305. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.975 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.822.992 [stars_engine.cc:1071] 619443 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.008 [stars_engine.cc:1017] 619443 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.027 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.052 [stars_engine.cc:1027] 619443 ProcReport: taskType=97. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.066 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.079 [stars_engine.cc:1570] 619443 ProcLogicCqReport: CQE warning, device_id=6, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.098 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.112 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.129 [task_info.cc:1614] 619443 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.140 [raw_device.cc:1305] 619443 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.160 [stars_engine.cc:1351] 619443 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.180 [npu_driver.cc:5618] 619443 GetSqEnable: deviceId=6, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.196 [stars_engine.cc:1365] 619443 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.209 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.224 [npu_driver.cc:5595] 619443 GetSqTail: deviceId=6, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.239 [npu_driver.cc:5406] 619443 SetSqHead: deviceId=6, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.256 [npu_driver.cc:5357] 619443 EnableSq: deviceId=6, tsId=0, sqId=0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.274 [stars_engine.cc:1430] 619443 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.299 [npu_driver.cc:5618] 619443 GetSqEnable: deviceId=6, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.313 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.329 [raw_device.cc:1331] 619443 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.823.344 [raw_device.cc:610] 619443 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.828.883 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.828.946 [ringbuffer_maintain_task.cc:98] 619443 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x1380 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.828.966 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.828.980 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.015 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.030 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.045 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.064 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.118 [stars_engine.cc:622] 619443 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=305, retCode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.136 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.154 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.169 [stream.cc:2886] 619443 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.184 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.197 [stream.cc:2886] 619443 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.209 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.228 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.244 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.256 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.276 [stars_engine.cc:437] 619443 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.292 [stream.cc:2886] 619443 StarsWaitForTask: No.3 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.345 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.376 [task_info.cc:111] 619443 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.392 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.407 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.425 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.439 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.456 [stars_engine.cc:1122] 619443 SyncTask: Task Wait: device_id=6, stream_id=0, task_id=2, logicCqId=305. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.576 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.593 [stars_engine.cc:1071] 619443 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.606 [stars_engine.cc:1017] 619443 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.623 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.634 [stars_engine.cc:1027] 619443 ProcReport: taskType=29. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.646 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.660 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.674 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.696 [npu_driver.cc:5618] 619443 GetSqEnable: deviceId=6, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.711 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.829.725 [raw_device.cc:1232] 619443 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.829.752 [atrace_api.c:40](tid:619443) AtraceCreate start. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.855 [trace_rb_log.c:45](tid:619443) init ring buffer time finished, realTime 1743644143829792139, monotonicTime 4211303973427, freq 99993 kHz, minutesWest 480 minutes [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.873 [trace_rb_log.c:137](tid:619443) [RUNTIME_ATRACE_DEV6_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.889 [tracer_mgr_operate.c:66](tid:619443) create object RUNTIME_ATRACE_DEV6_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.904 [atrace_api.c:44](tid:619443) AtraceCreate end. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.923 [trace_event.c:70](tid:619443) limitedNum 1, processType 0 [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.830.938 [trace_event.c:372](tid:619443) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.830.958 [runtime.cc:3159] 619443 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619443,python3.9):2025-04-03-09:35:43.830.983 [client_manager.cpp:196][SetProfilingCallback][tid:619443] [TsdClient] set profiling callback success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.013 [runtime.cc:3364] 619443 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.044 [context.cc:109] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.133 [context.cc:121] 619443 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.149 [context.cc:276] 619443 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.198 [raw_device.cc:803] 619443 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.216 [npu_driver_dcache_lock.cpp:39] 619443 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.355 [raw_device.cc:528] 619443 AllocStackPhyBase: AllocStackPhyBase device_id=6, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.407 [raw_device.cc:232] 619443 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.441 [runtime.cc:3393] 619443 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.457 [runtime.cc:3560] 619443 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.481 [api_impl.cc:2987] 619443 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.498 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.514 [api_impl.cc:2992] 619443 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:43.831.539 [awatchdog_core.c:175](tid:619443) create watchdog for id : 65543, tid : 619443, timeout : 300s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.831.555 [api_impl.cc:3025] 619443 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.831.579 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.831.596 [acl.cpp:121]619443 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.351 [api_impl.cc:3476] 619443 GetDeviceInfo: get device info, deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.392 [api_impl.cc:3476] 619443 GetDeviceInfo: get device info, deviceId=6 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.834.445 [device.cpp:133]619443 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 6, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.834.470 [device.cpp:154]619443 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.834.491 [context.cpp:87]619443 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.834.521 [context.cpp:98]619443 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.834.557 [stream.cpp:48]619443 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.589 [api_impl.cc:1120] 619443 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.652 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.702 [stream.cc:2246] 619443 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.745 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.834.763 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.192 [npu_driver.cc:6125] 619443 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.246 [npu_driver.cc:6137] 619443 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=306, streamId=2, logicCq=306, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.270 [npu_driver.cc:6189] 619443 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=2, logicCq=306. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.288 [npu_driver.cc:5916] 619443 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec0a9000, addr=dfffec0a9000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.304 [stream.cc:653] 619443 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.835.359 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.380 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.835.557 [stream.cc:291] 619443 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.836.991 [stream.cc:688] 619443 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=6, streamResId=65535 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.020 [stream_state_callback_manager.cc:45] 619443 Notify: not reg stream state callback func! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.037 [api_impl.cc:1162] 619443 StreamCreate: Succ, flags=1536, stream_id=2, context=aaab04034d70 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.837.055 [stream.cpp:189]619443 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.077 [stream.cc:3161] 619443 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.837.091 [stream.cpp:195]619443 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.837.146 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab05c0c0c0. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.837.183 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.837.198 [stream.cpp:48]619443 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.217 [api_impl.cc:1120] 619443 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.247 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.282 [stream.cc:2246] 619443 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.328 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.346 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.585 [npu_driver.cc:6125] 619443 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.631 [npu_driver.cc:6137] 619443 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=307, streamId=3, logicCq=307, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.653 [npu_driver.cc:6189] 619443 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=3, logicCq=307. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.671 [npu_driver.cc:5916] 619443 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec088000, addr=dfffec088000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.688 [stream.cc:653] 619443 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619443,python3.9):2025-04-03-09:35:43.837.734 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.837.754 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.838.074 [stream.cc:291] 619443 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.838.717 [stream.cc:688] 619443 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=6, streamResId=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.838.738 [stream_state_callback_manager.cc:45] 619443 Notify: not reg stream state callback func! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.838.750 [api_impl.cc:1162] 619443 StreamCreate: Succ, flags=1536, stream_id=3, context=aaab04034d70 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.838.765 [stream.cpp:189]619443 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.838.786 [stream.cc:3161] 619443 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.838.801 [stream.cpp:195]619443 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.838.837 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab05d3c720. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.838.974 [callback.cpp:25]619443 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471327465376. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.000 [npu_driver.cc:462] 619443 EventIdAlloc: event id alloc begin, device_id=6, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.018 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.059 [npu_driver.cc:483] 619443 EventIdAlloc: Event id alloc success, device_id=6, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.061 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.080 [event.cc:347] 619443 GenEventId: Event id alloc success, device_id=6, tsId=0, eventFlag=1, event_id=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.089 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.105 [subscribe.cc:54] 619443 AssignGroupId: give groupId=576 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.123 [npu_driver.cc:3775] 619443 SqCqAllocate: deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.151 [npu_driver.cc:3785] 619443 SqCqAllocate: alloc success, deviceId=6 sqId=91, cqId=91. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.158 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.182 [subscribe.cc:131] 619443 Insert: callback subscribe: groupId=576, threadId=281471327465376, devId=6, tsId=0, streamId=2, sqId=91, cqId=91 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.202 [callback.cpp:31]619443 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471327465376. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.212 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.839.234 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab05c0c0c0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.236 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.328 [callback.cpp:25]619443 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471319011232. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.352 [npu_driver.cc:462] 619443 EventIdAlloc: event id alloc begin, device_id=6, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.356 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.371 [npu_driver.cc:483] 619443 EventIdAlloc: Event id alloc success, device_id=6, ts_id=0, event_id=1, eventFlag=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.382 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.391 [event.cc:347] 619443 GenEventId: Event id alloc success, device_id=6, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.401 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.412 [subscribe.cc:54] 619443 AssignGroupId: give groupId=577 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.424 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.433 [npu_driver.cc:3775] 619443 SqCqAllocate: deviceId=6, tsId=0, groupId=577. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.442 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.459 [npu_driver.cc:3785] 619443 SqCqAllocate: alloc success, deviceId=6 sqId=92, cqId=92. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.478 [subscribe.cc:131] 619443 Insert: callback subscribe: groupId=577, threadId=281471319011232, devId=6, tsId=0, streamId=3, sqId=92, cqId=92 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.839.499 [callback.cpp:31]619443 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471319011232. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.507 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.839.529 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab05d3c720. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.839.556 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.839.622 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.839.961 [npu_driver.cc:2616] 619443 MemGetInfoEx: memory info type=1, type=2, free=8864755712, hug_free=56245616640 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.009 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.099 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.145 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.185 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.211 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62094M, Reserved MOC size for Other Components(HCCL/rts/etc.):3892M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58202M. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.840.236 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.840.270 [event.cpp:260]619443 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.840.293 [api_impl.cc:5713] 619443 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.368 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.389 [timeout_set_task.cc:115] 619443 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.405 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.419 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.440 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.452 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.468 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.482 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.519 [stars_engine.cc:622] 619443 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=305, retCode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.535 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.547 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.572 [stream.cc:2886] 619443 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.586 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.599 [stream.cc:2886] 619443 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.613 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.628 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.643 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.659 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.680 [stars_engine.cc:437] 619443 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.697 [stream.cc:2886] 619443 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.841.712 [event.cpp:262]619443 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.841.731 [event.cpp:269]619443 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.752 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.769 [timeout_set_task.cc:115] 619443 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.787 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.802 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.820 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.834 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.847 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.863 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.877 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.890 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.905 [stream.cc:2886] 619443 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.917 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.930 [stream.cc:2886] 619443 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.944 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.956 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.978 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.841.992 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.842.009 [stars_engine.cc:437] 619443 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.842.026 [stream.cc:2886] 619443 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.038 [event.cpp:277]619443 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.066 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.106 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.125 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 6 has been set [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.144 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.159 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.192 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.222 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619443,ffff964cc020,python3.9):2025-04-03-09:35:43.842.274 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.396 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.842.419 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.439 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.463 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:43.842.480 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.496 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.644 [acl.cpp:288]619443 aclInit: start to execute aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.672 [acl.cpp:298]619443 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.842.792 [acl.cpp:242]619443 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.851.861 [error_manager.cc:673]619443 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:43.852.637 [acl.cpp:315]619443 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:43.852.685 [prof_atls_plugin.cpp:210] (tid:619443) Module[45] register callback of ctrl handle. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.852.710 [ge_executor.cc:273]619443 Initialize:Init GeExecutor begin. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.852.759 [op_lib_register.cc:117]619443 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.852.846 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.852.885 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.852.919 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.852.953 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.852.984 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.014 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.044 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.075 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.105 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.135 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.173 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.203 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.233 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.263 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.292 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.323 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.352 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.382 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.412 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.853.442 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.466 [op_lib_register.cc:165]619443 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.500 [plugin_manager.cc:616]619443 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.517 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.547 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.570 [plugin_manager.cc:624]619443 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619443,python3.9):2025-04-03-09:35:43.853.587 [plugin_manager.cc:206]619443 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619443,python3.9):2025-04-03-09:35:43.854.384 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.429 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.446 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.458 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.854.505 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.854.851 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.886 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.903 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.916 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.854.953 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.854.994 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.019 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.032 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.046 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.065 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.405 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.438 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.454 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.469 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.505 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.846 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.879 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.897 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.912 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.855.950 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.855.987 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.003 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.017 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.032 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.064 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.417 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.452 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.470 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.485 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.523 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.864 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.896 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.912 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.925 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.856.961 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.856.997 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.013 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.028 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.045 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.062 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.400 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.432 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.462 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.475 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.509 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.848 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.881 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.897 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.912 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.857.947 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.857.984 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.001 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.015 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.031 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.046 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.384 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.417 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.435 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.449 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.484 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.833 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.876 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.893 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.908 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.858.943 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.858.989 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.859.006 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.859.023 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.038 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.055 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.067 [plugin_manager.cc:218]619443 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.087 [plugin_manager.cc:422]619443 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.103 [plugin_manager.cc:155]619443 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.124 [plugin_manager.cc:158]619443 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.138 [plugin_manager.cc:426]619443 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.471 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.859.523 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.859.540 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.859.554 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.595 [plugin_manager.cc:439]619443 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.616 [plugin_manager.cc:175]619443 ReversePathString:Enter ReversePathString schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.639 [plugin_manager.cc:187]619443 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.967 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:43.859.998 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:43.860.020 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619443,python3.9):2025-04-03-09:35:43.860.065 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619443,python3.9):2025-04-03-09:35:43.880.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.074 [op_tiling_registry.cc:162]619443 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.209 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.335 [op_tiling_registry.cc:162]619443 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.454 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.570 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.682 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.808 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.828 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.850 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619443,python3.9):2025-04-03-09:35:43.881.958 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.066 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.169 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.278 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.389 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.756 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.856 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619443,python3.9):2025-04-03-09:35:43.882.974 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.089 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.195 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.302 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.417 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.612 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.721 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.834 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.956 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619443,python3.9):2025-04-03-09:35:43.883.975 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.096 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.250 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.273 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.375 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.480 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.585 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.801 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619443,python3.9):2025-04-03-09:35:43.884.982 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.083 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.193 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.309 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.742 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.860 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.885.980 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.165 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.278 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.415 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.525 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.629 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.734 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.837 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.886.962 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.068 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.097 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.198 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.314 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.418 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.437 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.533 [op_tiling_registry.cc:122]619443 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.647 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.756 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.862 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619443,python3.9):2025-04-03-09:35:43.887.984 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.233 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.330 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.431 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.512 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.592 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.674 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.758 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.786 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.887 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.920 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.888.975 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.104 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.186 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.277 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.361 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.441 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.530 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.560 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.640 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.670 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.880 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.889.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.041 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.120 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.198 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.368 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.452 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.635 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.725 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.813 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.893 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.890.974 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.891.060 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.891.141 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.895.378 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619443,python3.9):2025-04-03-09:35:43.895.412 [op_tiling_registry.cc:148]619443 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619443,python3.9):2025-04-03-09:35:43.896.935 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.071 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.478 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.618 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.736 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.897.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.015 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.300 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.462 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.701 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.839 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.898.971 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.097 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.126 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.292 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.418 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.565 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.693 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.819 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.848 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.899.965 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.424 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.549 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.671 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.809 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.900.927 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.177 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.419 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.450 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.677 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.810 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.901.961 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.176 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.302 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.813 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.844 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.902.973 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.004 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.127 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.241 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.270 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.505 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.754 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.903.972 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.002 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.277 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.489 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.608 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.736 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.852 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.904.976 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.204 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.383 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.857 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.905.977 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.213 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.335 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.624 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.794 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.906.850 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff01345d70, registered count:1 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.034 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.058 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff0134e248, registered count:2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.313 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.465 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.588 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.718 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.842 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.907.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.072 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.249 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.410 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.648 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.908.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.014 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.147 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.262 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.385 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.549 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.610 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.636 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.793 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.909.916 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.033 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.150 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.269 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.411 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.439 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.571 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.687 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.910.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.148 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.265 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.518 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.631 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.755 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.785 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.911.905 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.395 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.515 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.630 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.757 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.874 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.912.998 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.913.120 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.913.237 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.913.458 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.913.584 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.913.715 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.311 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.357 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.526 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.654 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.916.897 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.013 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.132 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.256 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.374 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.530 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.666 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.698 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.825 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.857 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.917.973 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.918.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.918.207 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.918.330 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.918.570 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.918.864 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.919.024 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.919.154 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.919.284 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.919.407 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.920.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.920.966 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.921.299 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.921.776 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.922.396 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.922.530 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.922.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.922.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.922.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.923.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.925.583 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.926.446 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.926.623 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.926.789 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.927.810 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.927.975 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.928.147 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.928.689 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.928.815 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.928.951 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.071 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.195 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.332 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.461 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.700 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.929.963 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.127 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.251 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.570 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.704 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.823 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.930.944 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.105 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.292 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.419 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.546 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.576 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.770 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.931.894 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.037 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.270 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.397 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.570 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.695 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.823 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.949 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.932.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.099 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.129 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.250 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.370 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.399 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.534 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.655 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.786 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.933.976 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.934.106 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.934.322 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.934.480 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.934.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.934.865 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.057 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.263 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.526 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.647 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.935.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.006 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.126 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.300 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.553 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.674 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.789 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.936.951 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.937.201 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.938.499 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.941.035 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.941.247 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.941.466 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.941.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.941.986 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.942.153 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.196 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.331 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.491 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.661 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.811 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.943.976 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.944.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.944.341 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.944.499 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.944.622 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.944.719 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.032 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.066 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.129 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.156 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.183 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.209 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.236 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.264 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.317 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.356 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.410 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.437 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.466 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.491 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.571 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.678 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.704 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.730 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.756 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.832 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.884 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.912 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.939 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.974 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.945.998 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.024 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.078 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.108 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.134 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.161 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.188 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.216 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.242 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.267 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.321 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.348 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.374 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.402 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.454 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.479 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.504 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.528 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.700 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.946.875 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.082 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.110 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.138 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.165 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.317 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.436 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.551 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.670 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.947.913 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.081 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.243 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.415 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.548 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.671 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.702 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.732 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.888 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.948.914 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.033 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.204 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.388 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.547 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.667 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.862 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.949.988 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.199 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.321 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.446 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.502 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.680 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.950.863 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.024 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.183 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.539 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.665 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.951.839 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.016 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.248 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.396 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.527 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.777 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.952.901 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.067 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.237 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.356 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.600 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.953.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.954.324 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.954.803 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.954.925 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.051 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.081 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.218 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.337 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.588 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.708 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.836 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.955.967 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.095 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.298 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.424 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.553 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.751 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.956.875 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.016 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.136 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.341 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.653 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.773 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.957.900 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.038 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.159 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.283 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.401 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.565 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.687 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.958.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.959.134 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.959.340 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.959.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.959.719 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.959.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.470 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.526 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.550 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.587 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.613 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.640 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.960.666 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.961.444 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.961.877 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.962.306 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.962.503 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.962.721 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.963.142 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.963.278 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.963.459 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.966.289 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.968.118 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.968.334 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.968.479 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.968.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.968.900 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.078 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.280 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.638 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.673 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.701 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.729 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.755 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.961 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.969.992 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.166 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.347 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.524 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.703 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.914 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.970.947 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.971.123 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.971.299 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.971.477 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.971.674 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.971.862 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.028 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.060 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.263 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.474 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.507 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.623 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.658 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.783 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.814 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.951 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.972.981 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.234 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.369 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.505 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.624 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.740 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.973.947 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.974.166 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.974.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.974.966 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.975.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.975.546 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.975.758 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.975.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.067 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.103 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.266 [op_tiling_registry.cc:134]619443 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.299 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.471 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.506 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.976.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.977.707 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.182 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.345 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.511 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.677 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.709 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.978.865 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.184 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.217 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.395 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.574 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.735 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.979.918 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.980.074 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.980.539 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.980.905 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.981.157 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.981.359 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.982.442 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.982.903 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.983.475 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.983.641 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.983.808 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.983.932 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.323 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.361 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.481 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.513 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.877 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.984.998 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.985.167 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.985.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.985.477 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.985.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.985.999 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.155 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.397 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.581 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.765 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.986.981 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.987.157 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.987.187 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.987.365 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.987.559 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.989.132 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.994.911 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.994.973 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.997.852 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.998.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.998.445 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.998.785 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.998.999 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.999.206 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.999.392 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.999.560 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.999.756 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:43.999.904 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.000.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.000.318 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.000.453 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.000.616 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.118 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.292 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.458 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.492 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.614 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.756 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.001.952 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.151 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.358 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.659 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.002.808 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.003.479 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.003.714 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.003.897 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.025 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.233 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.266 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.418 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.539 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.568 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.595 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.754 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.004.973 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.068 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.322 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.523 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.554 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.784 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.005.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.684 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.789 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.815 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.888 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.971 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.008.995 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.078 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.234 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.255 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.378 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.401 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.420 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.442 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.626 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.009.648 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.010.832 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff01c98a10, registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.010.868 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff01c98c70, registered count:1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.011.296 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.011.792 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.013.521 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.013.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.013.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.014.174 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.014.633 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.073 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.215 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.504 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.661 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.046.801 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.047.441 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.047.538 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.047.800 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.047.892 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.048.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.048.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.048.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.048.927 [op_tiling_manager.cc:79]619443 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.048.991 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.217 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.263 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.329 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.386 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.427 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.479 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.049.527 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.544 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.581 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.604 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.626 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.650 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.050.979 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff341de9b0, registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.009 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.030 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.047 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.064 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.375 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff341e1da0, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.400 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.418 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.437 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.454 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.051.744 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.051.810 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.051.904 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.052.747 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff341982c0, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.052.783 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.052.803 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.052.822 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.052.841 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.175 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff3419c978, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.202 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.223 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.241 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.259 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.566 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.627 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.707 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.772 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.815 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.860 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.053.895 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.935 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.053.977 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.011 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.049 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.088 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.125 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.169 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.209 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.240 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.279 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.321 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.354 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.393 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.431 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.468 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.506 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.544 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.582 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.622 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.660 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.691 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.730 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.768 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.805 [op_tiling_manager.cc:73]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.842 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.880 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.054.912 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.950 [op_tiling_manager.cc:58]619443 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.054.991 [op_tiling_manager.cc:67]619443 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.022 [op_tiling_manager.cc:75]619443 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.041 [op_tiling_manager.cc:109]619443 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [201542] micro seconds. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.180 [plugin_manager.cc:639]619443 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.200 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.229 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.055.291 [host_cpu_engine.cc:254]619443 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.060.859 [host_cpu_engine.cc:273]619443 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619443,python3.9):2025-04-03-09:35:44.060.932 [host_cpu_engine.cc:254]619443 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.067.155 [host_cpu_engine.cc:267]619443 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619443,python3.9):2025-04-03-09:35:44.068.002 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619443] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.037 [folding.cc:404][InitCpuConstantFoldingNew][tid:619443]:"Init cpu constant folding begin." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.076 [folding.cc:408][InitCpuConstantFoldingNew][tid:619443]:"Number of registered ops: 234" [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.096 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Acos]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Acosh]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.132 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.147 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.208 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.228 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.246 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.272 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Add]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.286 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Addcdiv]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.301 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Addcmul]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.316 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[AffineGrid]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.332 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ApplyAdamV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.346 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Asin]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.364 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Asinh]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.380 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Assert]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.395 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Atan]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.407 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Atan2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.422 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Atanh]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.434 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[AvgPoolV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.448 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Betainc]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.460 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Bucketize]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.474 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CTCLossV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.487 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.500 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CacheSwapTable]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.512 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.527 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CaseCondition]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.539 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cast]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.552 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Ceil]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.566 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CheckNumerics]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.580 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ClipByValue]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.593 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ClipByValueV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.607 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Combinations]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.621 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CompareAndBitpack]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.634 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.648 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ConcatV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.664 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Conj]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.688 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ConjugateTranspose]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.702 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.717 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cosh]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.732 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[CropAndResize]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.747 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cross]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.760 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cummax]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.775 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cummin]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.791 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Cumsum]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.805 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Div]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.821 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[DynamicStitch]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.834 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EditDistance]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.851 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Elu]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EluGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.880 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.893 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.905 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.919 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.932 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.946 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.959 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.970 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.068.985 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.000 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.013 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.026 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.043 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.056 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.072 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingLookup]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.093 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.106 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.121 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.134 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.148 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.161 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Equal]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.177 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Expand]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.190 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ExpandDims]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.206 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Exponential]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.219 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.235 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Fill]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.249 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[FillDiagonal]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.264 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Fills]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.277 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Floor]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.290 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[FloorDiv]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.306 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[FloorMod]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.318 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GEMM]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.334 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GatherElements]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.347 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GatherV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.359 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Geometric]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.373 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GetDynamicDims]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.386 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Greater]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GreaterEqual]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.409 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GridSampler2D]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.421 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.434 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GridSampler3D]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.447 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.462 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Histogram]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.485 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.499 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Identity]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.512 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[IndexToAddr]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.527 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.539 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InitPartitionMap]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.550 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.563 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.577 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InvGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.589 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[InvertPermutation]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.603 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[IsClose]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.619 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[IsInf]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.632 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[IsNan]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.649 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Less]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[LinSpace]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.675 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[LinearSumAssignment]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.691 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Log]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.704 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Log1p]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.718 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.733 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.746 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.762 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MaskedFill]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.776 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MaskedScatter]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.791 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MaskedSelect]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.805 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.821 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MatMul]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.835 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.850 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MatrixDiagV3]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MinAreaPolygons]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.877 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Mul]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.898 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.912 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.926 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.940 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NLLLoss]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.956 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NLLLossGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.970 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NMSWithMask]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.983 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Neg]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.069.996 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.006 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NonZero]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.018 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.033 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.045 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[NotEqual]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.057 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[OnesLike]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.072 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[PSAMask]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.087 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[PSAMaskGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.100 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Pad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[PadD]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.129 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[PadV3]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.144 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[PadV3Grad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.158 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Pinverse]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.174 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Poisson]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.191 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Pow]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.206 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.220 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RNNTLoss]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.235 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RandomStandardNormal]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.247 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RandomUniform]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.260 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RealDiv]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.275 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ReduceAll]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.315 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ReduceMean]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.331 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ReduceSum]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.344 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Reshape]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.357 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Resize]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.369 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ResizeBilinear]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.383 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.398 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.413 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ResizeGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.427 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ResizeTrilinear]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.440 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ReverseSequence]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.453 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RightShift]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.466 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[RotatedNMS]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.492 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Round]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Rsqrt]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.520 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[STFT]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.548 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ScatterElements]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.560 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ScatterNdAdd]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.572 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ScatterNdMax]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ScatterNdMin]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.600 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SearchSorted]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.613 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SelectV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.626 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Sigmoid]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.639 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SigmoidGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.653 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Sign]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.667 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SilentCheck]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.681 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SilentCheckV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.698 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SilentCheckV3]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.722 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SliceWrite]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.738 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SoftmaxV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.756 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Sort]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.769 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.781 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SparseSegmentMean]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.797 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.812 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SparseSegmentSum]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.827 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SparseToDense]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.841 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SpatialTransformer]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.857 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SplitD]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.872 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SplitV]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.887 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Sqrt]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.902 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SqrtGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.917 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Square]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.931 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[SquaredDifference]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.945 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[StatelessBernoulli]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.958 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.971 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[StatelessRandperm]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.070.987 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[StridedSlice]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.001 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[StridedSliceV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.016 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Sub]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.030 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TableToResource]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.043 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Tan]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.059 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TanhGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.074 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TensorEqual]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.087 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.100 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ThreeNN]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Tile]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.137 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TileWithAxis]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.151 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TopK]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.166 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TopKPQDistance]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.178 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.195 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TopKV2D]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.209 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Trace]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.225 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TransData]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.239 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[TransDataRNN]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.253 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Tril]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.268 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Triu]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.300 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.314 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UninitPartitionMap]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.330 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UniqueConsecutive]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.344 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.358 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.372 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.386 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.400 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.429 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ViewCopy]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.445 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Voxelization]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.459 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[WarpAffine]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.475 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[ZerosLike]." [INFO] AICPU(619443,python3.9):2025-04-03-09:35:44.071.489 [folding.cc:413][InitCpuConstantFoldingNew][tid:619443]:"Register op[Zeta]." [INFO] OP(619443,python3.9):2025-04-03-09:35:44.071.507 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619443] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619443,python3.9):2025-04-03-09:35:44.071.521 [host_cpu_engine.cc:273]619443 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619443,python3.9):2025-04-03-09:35:44.071.612 [plugin_manager.cc:872]619443 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.075.043 [plugin_manager.cc:883]619443 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.112 [ops_kernel_executor_manager.cc:37]619443 Initialize:No library to load [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.132 [plugin_manager.cc:540]619443 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.146 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.181 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.203 [plugin_manager.cc:547]619443 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.075.220 [plugin_manager.cc:206]619443 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.113 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.192 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.214 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.230 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.274 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.667 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.702 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.719 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.732 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.768 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.818 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.834 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.847 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.864 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.897 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.913 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.076.928 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.076.943 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.077.336 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.370 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.387 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.399 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.077.435 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.077.833 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.865 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.881 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.896 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.077.932 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.976 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.077.992 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.007 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.078.021 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.052 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.068 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.082 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.078.098 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.078.491 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.523 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.539 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.078.554 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.078.590 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.078.982 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.014 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.030 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.042 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.079.075 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.120 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.136 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.150 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.079.166 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.196 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.211 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.226 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.079.241 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.079.629 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.661 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.678 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.079.690 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.079.724 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.115 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.146 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.200 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.225 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.271 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.306 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.321 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.336 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.352 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.381 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.395 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.409 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.425 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.773 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.807 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.823 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.080.838 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.080.878 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.277 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.309 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.326 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.339 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.389 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.424 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.439 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.452 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.465 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.498 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.513 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.081.525 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.539 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.551 [plugin_manager.cc:218]619443 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.572 [plugin_manager.cc:422]619443 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.585 [plugin_manager.cc:155]619443 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.606 [plugin_manager.cc:158]619443 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.081.623 [plugin_manager.cc:426]619443 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.011 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.082.046 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.082.063 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.082.078 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.117 [plugin_manager.cc:439]619443 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.138 [ge_executor.cc:259]619443 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.553 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.584 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.606 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.713 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.731 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.810 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.834 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.082.867 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.913 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.082.928 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.082.958 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.006 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.021 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.086 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.101 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.167 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.182 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.211 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.254 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.271 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.300 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.343 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.358 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.424 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.444 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.510 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.525 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.555 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.599 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.618 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.647 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.692 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.706 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.775 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.793 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.860 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.876 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.909 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.954 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.083.969 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.083.998 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.050 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.066 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.133 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.147 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.259 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.277 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.084.309 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.353 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.368 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.084.397 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.440 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.456 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.084.482 [opsproto_manager.cc:123]619443 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.521 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.579 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.595 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.609 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.624 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.084.639 [opsproto_manager.cc:147]619443 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.950 [operator_factory_impl.cc:215]619443 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619443,python3.9):2025-04-03-09:35:44.084.977 [operator_factory_impl.cc:269]619443 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619443,python3.9):2025-04-03-09:35:44.085.012 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.091.078 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619443,python3.9):2025-04-03-09:35:44.091.853 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.091.959 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.006 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.040 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.072 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.101 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.131 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.201 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.092.241 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.114.005 [operator_factory_impl.cc:368]619443 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619443,python3.9):2025-04-03-09:35:44.114.242 [operator_factory_impl.cc:284]619443 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.456 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.481 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.496 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.540 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.556 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619443,python3.9):2025-04-03-09:35:44.114.577 [operator_factory_impl.cc:254]619443 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619443,python3.9):2025-04-03-09:35:44.114.605 [operator_factory_impl.cc:340]619443 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619443,python3.9):2025-04-03-09:35:44.114.629 [operator_factory_impl.cc:307]619443 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.659 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.680 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.699 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.114.832 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.744 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.775 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.793 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.814 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.833 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.849 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.869 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.911 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.930 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.946 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.963 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.115.988 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.003 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.032 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.060 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.083 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.146 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.240 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.264 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.284 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.316 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.116.339 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.579 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.608 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.626 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.648 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.662 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.679 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.700 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.743 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.761 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.777 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.869 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.887 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.906 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.919 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.935 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.117.948 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.294 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.315 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.329 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.343 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.368 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.381 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.397 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.421 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.439 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.460 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.476 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.500 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.516 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.735 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.756 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.813 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.832 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.847 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.865 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.118.879 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.847 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.873 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.889 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.909 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.926 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.943 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.119.963 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.091 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.216 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.595 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.617 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.632 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.646 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.658 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.120.670 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.168 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.501 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.526 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.541 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.561 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.748 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.121.896 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.206 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.230 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.268 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.285 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.909 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.949 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.969 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.122.991 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.012 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.032 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.057 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.080 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.136 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.176 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.613 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.123.812 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.124.119 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.124.149 [operator_factory_impl.cc:222]619443 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.805 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.847 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.874 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.899 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.933 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.961 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.124.991 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.016 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.040 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.062 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.088 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.113 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.139 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.165 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.189 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.234 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.255 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.278 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.311 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.330 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.351 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.371 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.391 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.413 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.437 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.461 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.487 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.518 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.546 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.570 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.591 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.616 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.643 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.668 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.694 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.741 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.790 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.812 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.837 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.862 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.888 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.925 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.947 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.969 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.125.995 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.045 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.072 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.093 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.113 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.139 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.164 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.188 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.233 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.256 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.284 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.307 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.355 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.405 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.456 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.484 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.507 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.540 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.565 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.591 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.615 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.642 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.664 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.690 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.713 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.765 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.791 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.813 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.834 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.880 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.903 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.930 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.959 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.126.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.006 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.033 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.057 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.080 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.107 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.167 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.192 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.216 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.241 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.267 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.320 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.344 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.368 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.393 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.442 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.466 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.491 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.543 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.568 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.594 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.616 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.642 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.668 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.692 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.716 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.746 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.769 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.800 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.821 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.846 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.867 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.888 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.915 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.942 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.961 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.127.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.000 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.076 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.127 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.151 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.213 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.236 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.261 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.283 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.305 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.327 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.347 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.368 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.419 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.560 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.590 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.617 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.644 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.726 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.751 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.775 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.800 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.822 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.848 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.872 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.924 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.949 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.128.975 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.002 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.056 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.080 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.104 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.129 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.161 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.188 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.235 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.264 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.293 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.318 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.345 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.369 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.395 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.419 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.443 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.471 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.495 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.521 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.545 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.569 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.592 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.621 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.644 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.668 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.693 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.718 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.745 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.772 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.804 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.827 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.848 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.874 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.927 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.949 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.974 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.129.997 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.022 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.052 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.077 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.126 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.149 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.171 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.221 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.246 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.271 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.322 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.344 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.365 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.396 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.421 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.446 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.551 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.657 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.757 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.853 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.963 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.130.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.018 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.041 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.153 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.252 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.351 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.459 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.555 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.678 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.792 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.817 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.841 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.867 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.131.966 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.076 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.101 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.134 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.195 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.224 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.251 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.477 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.601 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.723 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.749 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.859 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.964 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.132.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.011 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.031 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.127 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.241 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.350 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.456 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.480 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.579 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.682 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.709 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.133.932 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.039 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.144 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.167 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.270 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.373 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.487 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.589 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.803 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.889 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.908 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.933 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.959 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.134.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.010 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.038 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.064 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.088 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.185 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.314 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.337 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.359 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.381 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.402 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.527 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.630 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.737 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.762 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.866 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.967 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.135.995 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.019 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.274 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.383 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.483 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.588 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.809 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.136.915 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.128 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.244 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.269 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.401 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.507 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.530 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.629 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.732 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.137.939 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.071 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.178 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.285 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.493 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.621 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.741 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.839 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.138.941 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.049 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.161 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.187 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.358 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.400 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.421 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.439 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.494 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.512 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.550 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.566 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.585 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.626 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.646 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.750 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.854 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.139.965 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.064 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.202 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.322 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.427 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.463 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.490 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.512 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.611 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.640 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.665 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.690 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.821 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.842 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.140.965 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.193 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.294 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.397 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.745 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.852 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.141.979 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.080 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.185 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.297 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.328 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.354 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.380 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.485 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.588 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.142.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.013 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.116 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.221 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.342 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.448 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.563 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.667 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.773 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.802 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.143.905 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.071 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.118 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.203 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.227 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.325 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.355 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.458 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.662 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.772 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.873 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.144.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.121 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.220 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.435 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.540 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.653 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.765 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.877 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.906 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.934 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.145.988 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.013 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.039 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.063 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.090 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.115 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.141 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.169 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.218 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.241 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.265 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.289 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.311 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.338 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.440 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.539 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.571 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.792 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.894 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.146.975 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.079 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.189 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.307 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.388 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.502 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.605 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.708 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.147.928 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.038 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.154 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.295 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.321 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.627 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.737 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.843 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.867 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.148.967 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.077 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.193 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.295 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.398 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.509 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.619 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.719 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.834 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.149.937 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.040 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.152 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.258 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.283 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.504 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.607 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.727 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.832 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.150.946 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.063 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.170 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.271 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.377 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.601 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.627 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.743 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.846 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.151.951 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.152.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.286 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.576 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.883 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.154.992 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.095 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.201 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.298 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.393 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.494 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.519 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.622 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.720 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.819 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.856 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.879 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.902 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.926 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.968 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.155.985 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.004 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.094 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.120 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.277 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.484 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.680 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.704 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.726 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.816 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.912 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.156.937 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.036 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.138 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.234 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.334 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.358 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.380 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.404 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.502 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.553 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.649 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.675 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.793 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.840 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.157.950 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.047 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.145 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.169 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.191 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.286 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.324 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.422 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.519 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.613 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.638 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.660 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.753 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.776 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.896 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.158.994 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.095 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.289 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.384 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.482 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.584 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.683 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.785 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.887 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.159.984 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.085 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.236 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.341 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.441 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.489 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.607 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.627 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.718 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.160.916 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.015 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.214 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.318 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.415 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.514 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.614 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.640 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.662 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.686 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.161.909 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.011 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.111 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.319 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.522 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.617 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.716 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.811 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.162.911 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.010 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.110 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.209 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.305 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.407 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.506 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.608 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.708 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.900 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.163.997 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.232 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.441 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.570 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.770 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.164.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.188 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.292 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.392 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.489 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.589 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.794 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.165.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.001 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.198 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.296 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.400 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.606 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.704 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.814 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.166.912 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.109 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.491 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.727 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.763 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.797 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.833 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.865 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.896 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.925 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.167.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.026 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.070 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.128 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.268 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.375 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.571 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.776 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.873 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.958 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.168.991 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.092 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.291 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.402 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.500 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.596 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.624 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.648 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.673 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.695 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.827 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.892 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.932 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.962 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.169.999 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.031 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.062 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.092 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.121 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.152 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.180 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.205 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.240 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.280 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.309 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.342 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.369 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.440 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.572 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.677 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.776 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.886 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.170.981 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.084 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.185 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.308 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.405 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.505 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.600 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.721 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.819 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.171.915 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.011 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.090 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.200 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.306 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.406 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.512 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.615 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.715 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.790 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.884 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.172.972 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.173 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.269 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.361 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.463 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.500 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.598 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.793 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.894 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.173.997 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.228 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.427 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.529 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.632 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.768 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.828 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.855 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.891 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.174.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.047 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.146 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.280 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.374 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.471 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.572 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.671 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.766 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.866 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.892 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.175.915 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.041 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.066 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.138 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.280 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.378 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.484 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.675 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.783 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.890 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.176.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.065 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.094 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.297 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.399 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.497 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.598 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.693 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.792 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.869 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.177.896 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.101 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.523 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.628 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.657 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.681 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.707 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.733 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.768 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.793 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.840 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.886 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.910 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.933 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.178.959 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.063 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.254 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.362 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.391 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.418 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.512 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.613 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.707 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.808 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.917 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.941 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.965 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.179.987 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.010 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.106 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.258 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.367 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.471 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.504 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.633 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.664 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.762 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.860 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.180.959 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.054 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.154 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.180 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.291 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.315 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.421 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.449 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.474 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.501 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.527 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.552 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.607 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.642 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.666 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.744 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.770 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.795 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.907 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.931 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.181.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.068 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.097 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.122 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.149 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.175 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.199 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.224 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.247 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.273 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.182.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.421 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.524 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.618 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.694 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.784 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.859 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.183.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.233 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.310 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.388 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.462 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.489 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.634 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.708 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.917 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.184.995 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.185.076 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.185.410 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.185.440 [opsproto_manager.cc:162]619443 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.185.487 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.185.533 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.111 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.168 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.189 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.268 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.285 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.346 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.363 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.376 [opp_so_manager.cc:56]619443 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.389 [plugin_manager.cc:540]619443 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.404 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.432 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.449 [plugin_manager.cc:547]619443 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.464 [plugin_manager.cc:206]619443 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.186.905 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.186.945 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.186.962 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.186.976 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.187.020 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.187.431 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.464 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.490 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.505 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.187.544 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.594 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.612 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.627 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.187.645 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.676 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.691 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.187.705 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.187.718 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.114 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.149 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.205 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.220 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.258 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.642 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.686 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.704 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.717 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.754 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.801 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.818 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.832 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.846 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.881 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.897 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.188.909 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.188.923 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.189.318 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.351 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.366 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.385 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.189.419 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.189.824 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.868 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.885 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.898 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.189.933 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.978 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.189.995 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.010 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.190.025 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.057 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.073 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.086 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.190.098 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.190.504 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.537 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.554 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.190.569 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.190.602 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.007 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.053 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.068 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.082 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.118 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.164 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.184 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.199 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.213 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.246 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.262 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.276 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.290 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.685 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.718 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.737 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.191.752 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.191.794 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.212 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.248 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.266 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.282 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.319 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.356 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.372 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.387 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.401 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.432 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.447 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.461 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.476 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.489 [plugin_manager.cc:218]619443 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.527 [plugin_manager.cc:422]619443 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.541 [plugin_manager.cc:155]619443 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.562 [plugin_manager.cc:158]619443 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.578 [plugin_manager.cc:426]619443 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.192.950 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.192.984 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.193.001 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.193.017 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.051 [plugin_manager.cc:439]619443 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.430 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.458 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.480 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.588 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.607 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.193.621 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.092 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.196.134 [op_impl_registry_holder_manager.cc:216]619443 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.204 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.251 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.740 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.855 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.875 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.196.889 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.206 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.245 [op_impl_registry_holder_manager.cc:216]619443 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.265 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.303 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.646 [op_impl_space_registry.cc:80]619443 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.669 [op_impl_space_registry.cc:83]619443 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.690 [op_impl_space_registry.cc:80]619443 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.706 [op_impl_space_registry.cc:83]619443 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.719 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.764 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.199.801 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.897 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.915 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.199.929 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.200.706 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.200.732 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.200.817 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.200.835 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.200.849 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.201.476 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.201.502 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.201.544 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.201.579 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.201.662 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.201.681 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.201.694 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.202.295 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.202.316 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.202.408 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.202.427 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.202.442 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.040 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.203.061 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.203.097 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.203.132 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.213 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.231 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.244 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.832 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.203.854 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.939 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.956 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.203.968 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.204.569 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.204.596 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.204.651 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.204.688 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.204.766 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.204.783 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.204.798 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.205.385 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.205.407 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619443,python3.9):2025-04-03-09:35:44.205.491 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.205.509 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.205.523 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.206.108 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.206.129 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.206.166 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.206.200 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.206.234 [opp_so_manager.cc:97]619443 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.206.312 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.206.355 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.206.370 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.206.382 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.238.854 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.238.923 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.238.944 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.238.964 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.238.984 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.002 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.022 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.041 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.059 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.078 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.115 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.131 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.150 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.168 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.186 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.206 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.224 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.243 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.260 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.278 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.297 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.315 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.352 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.370 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.406 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.425 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.444 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.464 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.482 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.501 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.520 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.539 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.557 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.577 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.616 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.635 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.652 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.667 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.702 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.722 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.756 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.777 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.814 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.834 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.857 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.877 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.896 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.916 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.934 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.951 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.970 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.239.987 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.006 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.024 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.043 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.062 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.078 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.097 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.116 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.134 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.155 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.232 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.252 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.271 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.310 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.345 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.363 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.381 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.408 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.424 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.443 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.461 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.479 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.514 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.532 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.549 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.568 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.586 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.622 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.641 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.657 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.675 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.694 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.711 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.727 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.745 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.762 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.801 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.854 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.875 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.901 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.919 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.936 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.971 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.240.991 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.009 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.026 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.040 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.059 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.079 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.101 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.122 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.140 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.160 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.181 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.204 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.242 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.264 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.311 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.331 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.349 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.371 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.390 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.435 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.453 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.496 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.520 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.540 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.559 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.600 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.621 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.638 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.655 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.681 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.701 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.722 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.834 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.858 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.880 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.899 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.920 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.941 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.241.981 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.001 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.046 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.136 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.156 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.178 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.198 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.219 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.239 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.259 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.280 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.325 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.346 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.365 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.383 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.403 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.438 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.477 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.494 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.511 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.532 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.553 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.576 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.599 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.629 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.671 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.690 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.712 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.736 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.757 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.778 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.797 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.817 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.855 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.880 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.899 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.919 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.939 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.964 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.242.987 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.007 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.028 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.071 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.110 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.135 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.157 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.175 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.206 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.225 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.245 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.265 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.286 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.310 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.330 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.351 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.371 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.406 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.429 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.643 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.743 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.840 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.243.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.025 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.273 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.380 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.577 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.831 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.855 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.876 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.244.967 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.067 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.090 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.111 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.133 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.153 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.176 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.199 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.298 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.395 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.505 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.631 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.726 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.823 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.848 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.872 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.245.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.000 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.103 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.208 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.305 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.343 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.439 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.558 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.655 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.763 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.856 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.958 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.246.986 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.081 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.179 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.284 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.388 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.493 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.586 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.612 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.632 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.653 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.672 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.694 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.716 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.738 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.759 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.780 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.801 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.822 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.247.926 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.026 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.051 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.074 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.123 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.145 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.403 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.501 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.524 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.623 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.722 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.746 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.768 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.868 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.248.970 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.077 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.173 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.374 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.480 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.580 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.783 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.923 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.249.946 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.049 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.152 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.176 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.270 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.370 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.573 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.677 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.702 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.807 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.250.909 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.106 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.131 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.239 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.336 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.433 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.537 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.636 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.786 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.889 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.926 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.972 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.251.993 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.014 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.035 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.057 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.079 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.123 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.201 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.225 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.247 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.270 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.291 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.392 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.491 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.589 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.698 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.252.904 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.182 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.206 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.226 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.244 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.337 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.360 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.383 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.403 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.609 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.708 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.817 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.253.916 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.011 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.319 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.418 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.513 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.609 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.715 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.740 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.761 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.785 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.884 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.254.992 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.187 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.289 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.396 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.495 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.594 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.700 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.798 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.255.914 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.014 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.112 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.136 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.271 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.399 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.422 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.445 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.488 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.509 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.620 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.648 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.749 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.846 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.256.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.070 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.169 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.375 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.398 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.494 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.595 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.693 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.788 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.257.997 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.099 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.123 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.145 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.169 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.212 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.233 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.296 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.317 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.338 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.360 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.411 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.451 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.474 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.496 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.701 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.724 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.828 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.258.928 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.023 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.094 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.197 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.300 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.404 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.475 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.572 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.776 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.875 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.259.977 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.074 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.211 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.322 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.353 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.463 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.560 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.666 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.763 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.869 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.893 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.260.995 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.196 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.297 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.394 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.499 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.694 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.800 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.897 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.261.999 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.206 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.230 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.335 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.433 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.543 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.649 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.750 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.849 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.262.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.064 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.166 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.266 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.368 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.475 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.501 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.600 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.714 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.810 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.263.908 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.009 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.357 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.456 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.552 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.656 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.866 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.264.967 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.072 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.169 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.295 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.406 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.508 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.554 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.576 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.619 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.640 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.662 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.683 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.802 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.824 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.265.929 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.035 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.139 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.237 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.336 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.360 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.381 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.475 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.572 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.596 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.798 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.266.902 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.004 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.030 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.056 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.080 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.175 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.199 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.219 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.317 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.340 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.362 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.463 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.487 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.509 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.527 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.627 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.725 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.823 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.847 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.868 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.972 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.267.995 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.103 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.236 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.340 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.365 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.400 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.498 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.521 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.543 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.638 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.738 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.845 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.268.947 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.046 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.147 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.254 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.369 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.470 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.572 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.765 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.866 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.269.978 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.083 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.183 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.208 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.229 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.324 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.347 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.571 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.667 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.859 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.270.958 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.055 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.155 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.345 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.402 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.423 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.616 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.720 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.819 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.271.914 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.118 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.250 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.354 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.449 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.641 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.736 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.837 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.272.941 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.042 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.139 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.238 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.336 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.438 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.532 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.633 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.730 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.830 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.273.923 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.054 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.152 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.256 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.360 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.453 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.556 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.664 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.760 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.863 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.274.957 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.051 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.154 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.257 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.361 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.456 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.552 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.660 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.757 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.856 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.275.957 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.051 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.201 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.397 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.493 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.586 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.681 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.897 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.924 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.949 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.276.976 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.001 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.025 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.125 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.145 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.166 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.214 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.234 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.328 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.525 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.622 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.721 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.822 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.277.923 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.130 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.228 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.422 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.521 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.621 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.644 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.712 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.843 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.871 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.899 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.930 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.278.977 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.004 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.029 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.052 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.077 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.100 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.124 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.146 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.176 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.205 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.229 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.250 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.272 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.352 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.455 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.478 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.575 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.668 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.770 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.864 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.279.971 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.073 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.232 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.334 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.432 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.525 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.548 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.651 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.750 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.855 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.280.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.028 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.108 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.219 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.323 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.432 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.538 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.639 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.715 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.804 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.890 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.281.991 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.100 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.202 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.285 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.414 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.525 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.628 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.731 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.842 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.282.942 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.155 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.262 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.360 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.465 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.564 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.687 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.712 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.735 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.759 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.780 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.824 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.283.926 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.142 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.272 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.481 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.576 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.679 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.802 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.824 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.284.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.008 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.090 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.189 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.387 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.487 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.588 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.707 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.807 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.907 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.285.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.004 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.102 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.204 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.320 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.422 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.523 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.628 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.737 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.808 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.832 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.286.940 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.042 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.144 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.245 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.343 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.368 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.391 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.437 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.478 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.497 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.520 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.542 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.604 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.624 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.730 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.835 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.934 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.958 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.287.993 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.086 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.233 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.331 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.435 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.555 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.583 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.605 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.721 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.818 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.288.919 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.031 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.054 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.074 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.176 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.198 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.298 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.399 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.507 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.613 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.730 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.754 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.849 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.872 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.289.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.025 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.047 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.068 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.090 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.111 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.135 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.156 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.178 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.200 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.244 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.266 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.287 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.382 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.407 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.429 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.532 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.554 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.575 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.596 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.618 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.639 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.656 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.705 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.727 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.854 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.290.922 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.047 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.122 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.190 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.262 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.656 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.735 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.811 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.895 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.291.962 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.031 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.146 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.281 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.390 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.463 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.533 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.597 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.617 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.292.882 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.302.030 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.306.775 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.329.434 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.329.756 [plugin_manager.cc:554]619443 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619443,python3.9):2025-04-03-09:35:44.329.779 [plugin_manager.cc:134]619443 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.329.818 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.329.836 [plugin_manager.cc:145]619443 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.329.850 [plugin_manager.cc:557]619443 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.329.863 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.329.888 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.195 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.242 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.265 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.322 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.340 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.393 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.410 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.422 [opp_so_manager.cc:66]619443 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.437 [plugin_manager.cc:616]619443 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.450 [plugin_manager.cc:108]619443 GetOppPath:Enter get opp path schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.476 [plugin_manager.cc:117]619443 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.495 [plugin_manager.cc:624]619443 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.511 [plugin_manager.cc:206]619443 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.848 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.330.887 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.330.905 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.330.918 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.330.967 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.331.384 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.418 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.434 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.446 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.331.482 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.532 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.549 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.331.563 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.331.578 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.331.594 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.331.991 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.023 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.051 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.066 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.332.103 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.332.514 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.550 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.567 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.583 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.332.621 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.659 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.676 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.332.692 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.332.709 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.332.725 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.115 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.148 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.165 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.178 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.212 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.606 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.639 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.657 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.671 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.708 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.745 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.762 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.333.776 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.793 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.333.808 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.193 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.225 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.241 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.256 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.291 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.684 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.715 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.732 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.745 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.796 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.833 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.850 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.334.864 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.881 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.334.894 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.288 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.319 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.335 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.350 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.382 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.779 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.810 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.826 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.841 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.876 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.910 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.933 [plugin_manager.cc:1026]619443 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.335.948 [plugin_manager.cc:322]619443 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.964 [plugin_manager.cc:212]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.980 [plugin_manager.cc:215]619443 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.335.995 [plugin_manager.cc:218]619443 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.016 [plugin_manager.cc:422]619443 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.030 [plugin_manager.cc:155]619443 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.052 [plugin_manager.cc:158]619443 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.068 [plugin_manager.cc:426]619443 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.465 [plugin_manager.cc:250]619443 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.336.504 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.336.522 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.336.536 [plugin_manager.cc:1063]619443 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.573 [plugin_manager.cc:439]619443 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.962 [plugin_manager.cc:996]619443 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619443,python3.9):2025-04-03-09:35:44.336.991 [plugin_manager.cc:1011]619443 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619443,python3.9):2025-04-03-09:35:44.337.012 [plugin_manager.cc:1015]619443 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.337.070 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.337.182 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.337.199 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.337.213 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.071 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff24a9c2c0, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.110 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.135 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.156 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.176 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.474 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff24aa0978, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.499 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.516 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.535 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.550 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.825 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.339.863 [op_impl_registry_holder_manager.cc:216]619443 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.879 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.339.922 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.340.236 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.340.287 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.340.387 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.340.404 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.340.417 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.341.954 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.341.989 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.007 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.026 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.045 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.322 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff24a4d9b0, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.346 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.363 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.379 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.409 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.683 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff24a50da0, registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.704 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.724 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.743 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.342.759 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.001 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.029 [op_impl_registry_holder_manager.cc:216]619443 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.047 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.082 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.399 [op_impl_space_registry.cc:84]619443 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.419 [op_impl_space_registry.cc:85]619443 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.431 [op_impl_space_registry.cc:86]619443 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.445 [op_impl_space_registry.cc:87]619443 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.458 [op_impl_space_registry.cc:93]619443 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.477 [op_impl_space_registry.cc:84]619443 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.492 [op_impl_space_registry.cc:85]619443 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.506 [op_impl_space_registry.cc:86]619443 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.522 [op_impl_space_registry.cc:87]619443 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.533 [op_impl_space_registry.cc:93]619443 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.548 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.612 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.651 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.343.688 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.784 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.801 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.343.813 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.344.407 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.344.433 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.344.474 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.344.567 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.344.583 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.344.596 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.345.248 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.345.272 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.345.336 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.345.374 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.345.410 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.345.501 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.345.518 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.345.532 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.345.980 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.000 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.040 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.131 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.147 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.163 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.747 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.768 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.806 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.851 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.346.887 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.975 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.346.992 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.347.004 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.347.450 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.347.472 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.347.510 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.347.598 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.347.614 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.347.626 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.348.247 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.273 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.314 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.351 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.394 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.348.484 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.348.501 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.348.514 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.348.937 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.958 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.348.997 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.085 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.101 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.113 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.687 [op_impl_space_registry.cc:240]619443 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.709 [opp_so_manager.cc:48]619443 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.747 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.782 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.814 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.854 [opp_so_manager.cc:126]619443 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.349.883 [opp_so_manager.cc:123]619443 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.964 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.349.981 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.350.024 [plugin_manager.cc:1126]619443 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.350.041 [plugin_manager.cc:1130]619443 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.350.055 [opp_so_manager.cc:45]619443 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.352.194 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.352.194 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.352.232 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.352.242 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.251 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.258 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.281 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.296 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.352.312 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.352.312 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.352.341 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.352.341 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.367 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.352.367 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.364.492 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.365.232 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.365.277 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.365.307 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.367.999 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.032 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.464 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.540 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.611 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.680 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.744 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.819 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.842 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.903 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.927 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.948 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.489.968 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.217 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.286 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.350 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.417 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.439 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.501 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.525 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.556 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.624 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.754 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.822 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.889 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.490.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.021 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.088 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.157 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.221 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.285 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.352 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.479 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.546 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.614 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.491.683 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.104 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.220 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.324 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.593 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.703 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.807 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.497.910 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.068 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.171 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.312 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.448 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.558 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.668 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.786 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.498.894 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.003 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.031 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.173 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.279 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.411 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.519 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.630 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.655 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.761 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.499.864 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.262 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.366 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.486 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.589 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.809 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.500.920 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.033 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.058 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.258 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.506 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.681 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffeee90d360, registered count:5 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.708 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.727 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.746 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.768 [op_check.cc:71]619443 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.789 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.501.916 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.034 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.234 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.407 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.515 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.541 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.676 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.784 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.895 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.502.920 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.032 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.151 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.258 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.582 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.609 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.503.831 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.020 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.142 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.284 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.514 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.720 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.504.888 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.037 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.184 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.301 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.533 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.644 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.505.764 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.035 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.202 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.226 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffeee997e08, registered count:7 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.395 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.426 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffeee9a0298, registered count:7 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.548 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.660 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.824 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.506.943 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.054 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.164 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.276 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.386 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.511 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.619 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.645 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.755 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.868 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.507.979 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.095 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.246 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.354 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.455 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.566 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.712 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.740 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.765 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.787 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.508.936 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.052 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.162 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.282 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.391 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.506 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.528 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.559 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.780 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.509.888 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.082 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.193 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.304 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.434 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.547 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.656 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.771 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.510.907 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.013 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.039 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.331 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.444 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.552 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.669 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.779 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.511.900 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.512.006 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.512.125 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.512.359 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.512.462 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.512.585 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.219 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.272 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.298 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.436 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.564 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.678 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.791 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.515.902 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.014 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.119 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.281 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.400 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.428 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.555 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.666 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.796 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.516.911 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.025 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.152 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.377 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.665 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.820 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.517.932 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.518.052 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.518.160 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.518.831 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.519.646 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.519.979 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.520.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.067 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.192 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.320 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.577 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.521.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.524.076 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.524.945 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.525.114 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.525.266 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.526.268 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.526.425 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.526.577 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.179 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.284 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.526 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.645 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.752 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.861 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.527.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.099 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.255 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.411 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.517 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.629 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.814 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.528.935 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.043 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.161 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.320 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.500 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.609 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.742 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.529.938 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.165 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.332 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.452 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.620 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.741 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.856 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.963 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.530.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.099 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.124 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.242 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.355 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.379 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.483 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.596 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.711 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.531.904 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.020 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.240 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.390 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.731 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.532.912 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.107 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.223 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.335 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.446 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.561 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.585 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.533.892 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.001 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.106 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.213 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.336 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.453 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.592 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.534.821 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.535.860 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.538.612 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.538.826 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.539.033 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.539.367 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.539.533 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.539.684 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.540.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.540.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.540.830 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.540.987 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.138 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.265 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.429 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.586 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.872 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.541.986 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.077 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.397 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.425 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.450 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.497 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.523 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.546 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.568 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.591 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.614 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.635 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.654 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.677 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.697 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.718 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.738 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.760 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.803 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.826 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.856 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.879 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.899 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.919 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.941 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.960 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.542.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.006 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.028 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.073 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.098 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.118 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.136 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.165 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.187 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.209 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.230 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.251 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.272 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.312 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.356 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.377 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.408 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.430 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.452 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.496 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.520 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.540 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.560 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.581 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.603 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.543.929 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.091 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.140 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.197 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.222 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.364 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.473 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.583 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.688 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.791 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.544.898 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.057 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.174 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.333 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.450 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.574 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.599 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.622 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.731 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.756 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.781 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.545.894 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.053 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.208 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.354 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.664 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.774 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.546.955 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.066 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.187 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.212 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.237 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.695 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.547.853 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.036 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.225 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.338 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.492 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.648 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.826 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.548.953 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.065 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.174 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.290 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.406 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.555 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.710 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.821 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.549.944 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.550.055 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.550.178 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.550.288 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.550.734 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.168 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.295 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.411 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.436 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.547 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.663 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.773 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.551.895 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.002 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.124 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.268 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.373 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.524 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.641 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.754 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.552.938 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.050 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.170 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.279 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.460 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.644 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.761 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.884 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.553.992 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.101 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.210 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.432 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.585 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.698 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.554.854 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.555.125 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.555.319 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.555.480 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.555.667 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.555.885 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.087 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.326 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.350 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.370 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.416 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.438 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.459 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.556.481 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.557.247 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.557.651 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.558.064 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.558.263 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.558.461 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.558.878 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.558.998 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.559.166 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.561.718 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.563.562 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.563.681 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.563.799 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.563.920 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.246 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.405 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.579 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.912 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.943 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.968 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.564.989 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.012 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.194 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.221 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.384 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.545 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.714 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.565.885 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.060 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.086 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.243 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.420 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.587 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.750 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.566.925 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.075 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.101 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.253 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.449 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.476 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.590 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.709 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.567.739 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.117 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.145 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.304 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.410 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.522 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.643 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.746 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.568.855 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.569.042 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.569.241 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.569.631 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.570.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.570.197 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.570.531 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.570.720 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.570.913 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.037 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.157 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.307 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.347 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.372 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.571.398 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.572.527 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.572.974 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.150 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.303 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.457 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.485 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.638 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.794 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.945 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.573.973 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.574.132 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.574.288 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.574.432 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.574.602 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.574.747 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.575.137 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.575.489 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.575.731 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.575.910 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.577.065 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.577.535 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.121 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.288 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.587 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.753 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.909 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.578.940 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.069 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.096 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.429 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.544 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.703 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.863 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.579.988 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.580.338 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.580.530 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.580.691 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.580.806 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.580.919 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.080 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.265 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.453 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.480 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.675 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.581.829 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.582.019 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.583.513 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.589.362 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.589.414 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.591.983 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.592.319 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.592.625 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.592.968 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.593.175 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.593.389 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.593.557 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.593.717 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.593.906 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.594.048 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.594.143 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.594.399 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.594.524 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.594.686 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.177 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.341 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.508 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.538 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.656 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.782 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.595.968 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.204 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.421 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.584 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.696 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.812 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.596.837 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.039 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.238 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.390 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.508 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.673 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.700 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.836 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.954 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.597.980 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.005 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.156 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.364 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.454 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.695 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.889 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.917 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.598.942 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.599.140 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.599.168 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.276 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.363 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.399 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.467 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.553 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.575 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.647 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.813 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.834 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.954 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.976 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.602.993 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.603.012 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.603.176 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.603.196 [tuning_tiling_registry.cc:30]619443 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.604.410 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffeef2dc0b0, registered count:7 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.604.447 [op_check.cc:24]619443 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffeef2dc310, registered count:5 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.604.853 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.605.349 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.607.024 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.607.200 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.607.283 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.607.650 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.608.099 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.638.917 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.639.038 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.639.229 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.639.328 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.639.467 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.639.609 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.640.348 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.640.429 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.640.683 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.640.770 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.640.909 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.641.135 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.641.318 [op_impl_registry.cc:122]619443 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619443,python3.9):2025-04-03-09:35:44.642.012 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.647.697 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619443,python3.9):2025-04-03-09:35:44.649.837 [op_impl_registry_holder_manager.cc:208]619443 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.727.137 [op_impl_space_registry.cc:109]619443 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.728.491 [op_impl_space_registry.cc:84]619443 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.728.517 [op_impl_space_registry.cc:85]619443 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.728.531 [op_impl_space_registry.cc:86]619443 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.728.545 [op_impl_space_registry.cc:87]619443 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.728.559 [op_impl_space_registry.cc:93]619443 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619443,python3.9):2025-04-03-09:35:44.729.562 [op_impl_space_registry.cc:273]619443 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.729.822 [plugin_manager.cc:564]619443 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619443,python3.9):2025-04-03-09:35:44.729.845 [plugin_manager.cc:134]619443 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.729.887 [file_utils.cc:47]619443 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.729.904 [plugin_manager.cc:145]619443 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619443,python3.9):2025-04-03-09:35:44.729.935 [plugin_manager.cc:567]619443 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619443,python3.9):2025-04-03-09:35:44.729.980 [mem_manager.cc:31]619443 Initialize:MemManager Initialize. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.013 [rts_caching_mem_allocator.cc:131]619443 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.030 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.042 [graph_mem_allocator.cc:17]619443 Initialize:MemoryAllocator::Initialize. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.058 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.074 [graph_mem_allocator.cc:17]619443 Initialize:MemoryAllocator::Initialize. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.086 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.101 [caching_allocator.cc:391]619443 FreeBlocks:Free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.113 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.127 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.142 [caching_allocator.cc:391]619443 FreeBlocks:Free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.152 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.165 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.180 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.193 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.208 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.220 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.234 [session_scope_mem_allocator.cc:73]619443 FreeAllMemory:Free all memory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.248 [mem_manager.cc:102]619443 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.262 [session_scope_mem_allocator.cc:73]619443 FreeAllMemory:Free all memory [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.301 [dump_manager.cc:152]619443 Init:exception dump is not enabled [INFO] GE(619443,python3.9):2025-04-03-09:35:44.730.339 [ge_executor.cc:315]619443 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.358 [acl.cpp:321]619443 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.373 [dump.cpp:841]619443 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.481 [dump.cpp:759]619443 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.496 [dump.cpp:594]619443 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.506 [dump.cpp:224]619443 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.539 [dump.cpp:703]619443 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.557 [dump.cpp:771]619443 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.730.579 [dump.cpp:703]619443 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.645 [adx_datadump_server.cpp:77][tid:619443] start to do dump init [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.664 [adx_dump_record.cpp:210][tid:619443] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.681 [adx_dump_record.cpp:214][tid:619443] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.835 [adx_server_manager.cpp:149][tid:619675] server manager add component (10) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.835 [adx_dump_record.cpp:627][tid:619676] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.730.863 [adx_server_manager.cpp:165][tid:619675] server manager components init successfully [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.006 [adx_server_manager.cpp:203][tid:619677] Run Server(0) Process [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.062 [adx_server_manager.cpp:369][tid:619677] device up 0 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.082 [hdc_comm_opt.cpp:59][tid:619677] open device[0] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.186 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.205 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.222 [hdc_api.cpp:119][tid:619677] logDevId 0 create HDC server successfully [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.253 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.269 [adx_server_manager.cpp:369][tid:619677] device up 1 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.281 [hdc_comm_opt.cpp:59][tid:619677] open device[1] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.350 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.366 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.378 [hdc_api.cpp:119][tid:619677] logDevId 1 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.393 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.405 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.419 [adx_server_manager.cpp:369][tid:619677] device up 2 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.429 [hdc_comm_opt.cpp:59][tid:619677] open device[2] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.495 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.512 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.524 [hdc_api.cpp:119][tid:619677] logDevId 2 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.537 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.565 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.577 [adx_server_manager.cpp:369][tid:619677] device up 3 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.589 [hdc_comm_opt.cpp:59][tid:619677] open device[3] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.656 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.673 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.686 [hdc_api.cpp:119][tid:619677] logDevId 3 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.698 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.714 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.726 [adx_server_manager.cpp:369][tid:619677] device up 4 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.738 [hdc_comm_opt.cpp:59][tid:619677] open device[4] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.831 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.848 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.859 [hdc_api.cpp:119][tid:619677] logDevId 4 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.871 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.886 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.900 [adx_server_manager.cpp:369][tid:619677] device up 5 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.731.910 [hdc_comm_opt.cpp:59][tid:619677] open device[5] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.973 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.731.990 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.004 [hdc_api.cpp:119][tid:619677] logDevId 5 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.018 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.030 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.042 [adx_server_manager.cpp:369][tid:619677] device up 6 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.056 [hdc_comm_opt.cpp:59][tid:619677] open device[6] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.732.145 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.732.172 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.190 [hdc_api.cpp:119][tid:619677] logDevId 6 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.202 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.220 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.244 [adx_server_manager.cpp:369][tid:619677] device up 7 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.254 [hdc_comm_opt.cpp:59][tid:619677] open device[7] server[13] [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.732.321 [drv_log_user.c:623][ascend][curpid:619443,619677][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.732.338 [hdc_server.c:363][ascend][curpid:619443,619677][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.352 [hdc_api.cpp:119][tid:619677] logDevId 7 create HDC server successfully [WARNING] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.363 [adx_hdc_epoll.cpp:34][tid:619677] hdc epoll has been created. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.732.379 [adx_server_manager.cpp:111][tid:619677] create server info [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.733.937 [adx_server_manager.cpp:457][tid:619443] The server is initialized after waiting 3 times. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.733.958 [ge_executor.cc:1349]619443 SetDump:Start to set dump config [INFO] GE(619443,python3.9):2025-04-03-09:35:44.733.972 [dump_manager.cc:201]619443 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.734.000 [dump_manager.cc:261]619443 SetExceptionDump:Enable brief exception dump. [INFO] GE(619443,python3.9):2025-04-03-09:35:44.734.028 [ge_executor.cc:1355]619443 SetDump:Set dump config successfully [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.734.060 [dump_manager.cpp:146][tid:619443] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:44.734.901 [dump_manager.cpp:301][tid:619443] Register exception callback, registered: 1 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.734.921 [dump.cpp:825]619443 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.734.943 [acl.cpp:327]619443 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.734.958 [acl.cpp:329]619443 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.734.970 [acl_resource_manager.cpp:248]619443 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.057 [acl_resource_manager.cpp:275]619443 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.075 [acl_resource_manager.cpp:278]619443 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.087 [acl.cpp:335]619443 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.099 [acl.cpp:336]619443 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.163 [json_parser.cpp:260]619443 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.181 [acl.cpp:343]619443 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:44.735.200 [prof_atls_plugin.cpp:210] (tid:619443) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.216 [acl.cpp:351]619443 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.226 [profiling.cpp:141]619443 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.240 [profiling.cpp:135]619443 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.311 [profiling.cpp:167]619443 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619443.json], errorCode = 0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.329 [profiling.cpp:172]619443 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619443.json] is empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.340 [profiling.cpp:176]619443 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.353 [profiling.cpp:179]619443 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.367 [profiling.cpp:92]619443 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:44.735.647 [ai_drv_dev_api.cpp:316] >>> (tid:619443) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:44.735.735 [msprof_callback_impl.cpp:132] >>> (tid:619443) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.753 [profiling.cpp:122]619443 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.764 [profiling.cpp:186]619443 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.777 [hash_utils.cpp:108]619443 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619443.json [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.814 [hash_utils.cpp:128]619443 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619443.json, hash: 3917491750093904922 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.830 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.859 [acl.cpp:280]619443 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.874 [acl.cpp:385]619443 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.735.891 [runtime.cc:5394] 619443 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.735.907 [api_impl.cc:7407] 619443 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.735.924 [api_impl.cc:4211] 619443 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.735.941 [acl.cpp:420]619443 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.736.062 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.736.202 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.736.261 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.736.285 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.740195] [info] [619443] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist mki_log delete old file:/home/jenkins/atb/log/atb_1952516_20250328090703.log [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:44.740.847 [prof_atls_plugin.cpp:210] (tid:619443) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.741.748 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.741.775 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.741.851 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 6, rank size = 8. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.741.871 [device.cpp:143]619443 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.897 [api_impl.cc:2999] 619443 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.915 [device.cc:306] 619443 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.933 [runtime.cc:3393] 619443 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.949 [runtime.cc:3560] 619443 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.966 [api_impl.cc:2987] 619443 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.741.983 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.742.000 [api_impl.cc:2992] 619443 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.742.022 [api_impl.cc:3025] 619443 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.742.038 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.742.051 [acl.cpp:121]619443 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.742.075 [device.cpp:154]619443 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.742.127 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.742.148 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is _npu_log, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.742.168 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.742.181 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.742.197 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.742.225 [queue_client.c:151][ascend][curpid:619443,619443][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.742.410 [queue_client.c:305][ascend][curpid:619443,619443][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=6; vfid=0; hostpid=619443; master_pid=619443; proc_type=0; devpid=30773). [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.748.760 [queue_client.c:362][ascend][curpid:619443,619443][drv][queuemng][QueueInit]QueueInit device. (devid=6; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.748.975 [npu_driver.cc:4114] 619443 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.748.994 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.011 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=_npu_log. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.749.272 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.292 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.400 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.429 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.749.453 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.463 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.484 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.481 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.509 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.523 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.537 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.576 [npu_driver.cc:4087] 619443 MemQueueInit: repeated init [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.592 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.606 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=ms_tensor_dump. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.749.794 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.815 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.914 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.749.940 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.944 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.749.955 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.968 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.990 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.749.995 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.004 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.018 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.055 [npu_driver.cc:4087] 619443 MemQueueInit: repeated init [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.071 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.085 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=ms_tensor_summary. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.750.249 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.269 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.431 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.455 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.750.463 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.476 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.480 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.493 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.500 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.515 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.527 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.568 [npu_driver.cc:4087] 619443 MemQueueInit: repeated init [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.583 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.597 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=ms_image_summary. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.750.761 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.780 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.864 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.896 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.750.897 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.916 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.750.922 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.933 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.942 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.955 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.750.970 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.005 [npu_driver.cc:4087] 619443 MemQueueInit: repeated init [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.021 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.032 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=ms_scalar_summary. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.751.182 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.203 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.289 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.313 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.751.319 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.330 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.337 [tensor_data_transfer.cpp:898]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.344 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.356 [api_impl.cc:6077] 619443 MemQueueInit: Start to init queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.370 [npu_driver.cc:4076] 619443 MemQueueInit: init mem queue, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.385 [npu_driver.cc:4026] 619443 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.419 [npu_driver.cc:4087] 619443 MemQueueInit: repeated init [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.435 [api_impl.cc:6040] 619443 MemQueueCreate: Start to create queue on devId 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.448 [npu_driver.cc:3931] 619443 MemQueueCreate: create mem queue, deviceId=6, name=ms_histogram_summary. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.751.607 [queue_interface.c:182][ascend][curpid:619443,619443][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.626 [tensor_data_transfer.cpp:940]619443 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 6, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.789 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.751.817 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.751.831 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.837 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.751.856 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.751.961 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.751.994 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.011 [plugin_manager.cc:25] [619443]hcom ops plugin init start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.046 [plugin_manager.cc:38] [619443]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.067 [plugin_manager.cc:42] [619443]hcom running normal mode. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.103 [hcom_plugin.cc:50] [619443]Initialize start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.118 [hcom_plugin.cc:52] [619443]initializeCount_ : 1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.147 [hcom_plugin.cc:60] [619443]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.206 [hcom_plugin.cc:63] [619443]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.229 [hcom_plugin.cc:329] [619443]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.246 [hcom_plugin.cc:281] [619443]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:44.752.269 [hcom_plugin.cc:217] [619443]option profiling mode is not found. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.286 [hcom_plugin.cc:164] [619443][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.300 [hcom_plugin.cc:180] [619443][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.312 [hcom_plugin.cc:244] [619443]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.405 [hcom_plugin.cc:82] [619443]hccl ops plugin init success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.422 [plugin_manager.cc:48] [619443]hcom ops plugin init end. ret[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.433 [hcom_plugin.cc:126] [619443]get hccl kernel info store start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.453 [hcom_plugin.cc:132] [619443]get hccl kernel info store finished. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.752.483 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.752.516 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.530 [hcom_executor.cc:29] [619443]Hcom Excutor Initialize start. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.585 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.603 [adapter_rts.cc:234] [619443][hrtGetDevice]deviceLogicId[6] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.634 [api_c_soc.cc:70] 619443 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.667 [hcom_common.cc:85] [619443][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.693 [hcom_common.cc:88] [619443][HcomGetCurHcomCtx] is no Used deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.716 [hcom.cc:486] [619443]Get Hcom Init Status: [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.732 [hcom_common.cc:85] [619443][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.752.745 [hcom_rpc.cc:1019] [619443]Get Hcom Init Status: [0] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:44.752.769 [hcom_base_funs.cc:36] [619443][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:44.752.793 [hcom_executor.cc:121] [619443][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.828 [api_impl.cc:1120] 619443 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.852 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.897 [stream.cc:2246] 619443 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.946 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.752.961 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.285 [npu_driver.cc:6125] 619443 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.336 [npu_driver.cc:6137] 619443 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=308, streamId=4, logicCq=308, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.358 [npu_driver.cc:6189] 619443 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=4, logicCq=308. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.377 [npu_driver.cc:5916] 619443 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec067000, addr=dfffec067000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.392 [stream.cc:653] 619443 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.753.445 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.464 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.753.778 [stream.cc:291] 619443 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.376 [stream.cc:688] 619443 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=6, streamResId=65535 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.754.406 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.754.418 [acl_resource_manager.cpp:816]619443 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.433 [api_impl.cc:1162] 619443 StreamCreate: Succ, flags=1536, stream_id=4, context=aaab04034d70 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.754.463 [stream.cc:73] [619443]Construct stream by stream type success, ptr[0xaaab0ffc9980] ctx[0xaaab04034d70], stream id[4], cqId[4], logicCqid[308] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.502 [api_impl.cc:1120] 619443 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.526 [npu_driver.cc:404] 619443 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.557 [stream.cc:2246] 619443 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.595 [stream_sqcq_manage.cc:147] 619443 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.612 [npu_driver.cc:882] 619443 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.838 [npu_driver.cc:6125] 619443 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.881 [npu_driver.cc:6137] 619443 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=309, streamId=5, logicCq=309, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.905 [npu_driver.cc:6189] 619443 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=5, logicCq=309. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.924 [npu_driver.cc:5916] 619443 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec046000, addr=dfffec046000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.754.940 [stream.cc:653] 619443 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619443,python3.9):2025-04-03-09:35:44.754.987 [devmm_svm.c:1972][ascend][curpid:619443,619443][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.755.005 [npu_driver.cc:6080] 619443 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.755.307 [stream.cc:291] 619443 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.755.943 [stream.cc:688] 619443 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=6, streamResId=65535 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.755.964 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.755.977 [acl_resource_manager.cpp:816]619443 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.755.992 [api_impl.cc:1162] 619443 StreamCreate: Succ, flags=1536, stream_id=5, context=aaab04034d70 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.756.015 [stream.cc:73] [619443]Construct stream by stream type success, ptr[0xaaab100459e0] ctx[0xaaab04034d70], stream id[5], cqId[5], logicCqid[309] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:44.756.133 [sal.cc:619] [619707]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.756.178 [hcom_executor.cc:587] [619707]MessageProcessThreadLoop start queType:[0] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:44.756.197 [sal.cc:619] [619708]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.206 [api_impl.cc:2999] 619707 SetDevice: device_id=6, deviceMode=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.756.227 [hcom_executor.cc:587] [619708]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.244 [device.cc:306] 619707 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.259 [runtime.cc:3393] 619707 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.260 [api_impl.cc:2999] 619708 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.272 [api_impl.cc:2999] 619709 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.276 [runtime.cc:3560] 619707 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.280 [device.cc:306] 619708 Init: device mode has been already set, mode=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:44.756.287 [hcom_executor.cc:34] [619443][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.300 [api_impl.cc:2987] 619707 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.309 [runtime.cc:3393] 619708 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.310 [device.cc:306] 619709 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.319 [runtime.cc:5239] 619707 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.327 [runtime.cc:3560] 619708 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.324 [api_impl.cc:2999] 619710 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.332 [runtime.cc:3393] 619709 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.332 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.341 [api_impl.cc:2992] 619707 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.344 [api_impl.cc:2987] 619708 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.353 [device.cc:306] 619710 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.361 [runtime.cc:3560] 619709 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.375 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:44.756.379 [awatchdog_core.c:175](tid:619707) create watchdog for id : 65543, tid : 619707, timeout : 300s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.383 [runtime.cc:5239] 619708 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.387 [runtime.cc:3393] 619710 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.394 [api_impl.cc:2987] 619709 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.404 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.404 [api_impl.cc:3025] 619707 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.417 [api_impl.cc:2992] 619708 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.418 [runtime.cc:3560] 619710 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.425 [runtime.cc:5239] 619709 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.444 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:44.756.463 [awatchdog_core.c:175](tid:619708) create watchdog for id : 65543, tid : 619708, timeout : 300s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.463 [api_impl.cc:2987] 619710 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.473 [api_impl.cc:2992] 619709 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.486 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.483 [api_impl.cc:3025] 619708 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.492 [runtime.cc:5239] 619710 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:44.756.501 [awatchdog_core.c:175](tid:619709) create watchdog for id : 65543, tid : 619709, timeout : 300s [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.521 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1162.53 msec. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.518 [api_impl.cc:2992] 619710 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.521 [api_impl.cc:3025] 619709 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:44.756.541 [awatchdog_core.c:175](tid:619710) create watchdog for id : 65543, tid : 619710, timeout : 300s [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.547 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.756.560 [api_impl.cc:3025] 619710 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [WARNING] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.572 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.618 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.013 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.657 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.692 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.716 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.074 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.827 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:44.756.855 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:44.756.930 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:44.756.960 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.756.999 [device.cpp:143]619711 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.028 [api_impl.cc:2999] 619711 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.044 [device.cc:306] 619711 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.056 [runtime.cc:3393] 619711 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.072 [runtime.cc:3560] 619711 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.085 [api_impl.cc:2987] 619711 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.100 [runtime.cc:5239] 619711 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.117 [api_impl.cc:2992] 619711 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:44.757.136 [awatchdog_core.c:175](tid:619711) create watchdog for id : 65543, tid : 619711, timeout : 300s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.757.153 [api_impl.cc:3025] 619711 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.757.166 [acl.cpp:115]619711 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.757.181 [acl.cpp:121]619711 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.757.192 [device.cpp:154]619711 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:44.757.227 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.218 msec. [WARNING] DEVICE(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:44.757.456 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 1 [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.868.176 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.868.180 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.868.203 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:44.868.208 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.221 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.239 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.245 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.254 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.868.263 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.868.268 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.868.279 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:44.868.282 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.296 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:44.868.300 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.036.747 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.036.772 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.036.790 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.036.803 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.036.826 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.036.911 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:45.257.683 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:45.257.717 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.458 msec. [WARNING] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:45.257.738 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:45.257.762 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.257.889 [device.cpp:143]619761 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.257.925 [api_impl.cc:2999] 619761 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.257.942 [device.cc:306] 619761 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.257.956 [runtime.cc:3393] 619761 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.257.980 [runtime.cc:3560] 619761 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.257.992 [api_impl.cc:2987] 619761 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.258.006 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.258.025 [api_impl.cc:2992] 619761 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:45.258.045 [awatchdog_core.c:175](tid:619761) create watchdog for id : 65543, tid : 619761, timeout : 300s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.258.062 [api_impl.cc:3025] 619761 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.258.074 [acl.cpp:115]619761 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.258.089 [acl.cpp:121]619761 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.258.101 [device.cpp:154]619761 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.308.348 [device.cpp:231]619761 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.380 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.564 [api_c_soc.cc:70] 619761 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.308.620 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.641 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.655 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.667 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.679 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.693 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=0. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.308.711 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=0.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.308.740 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.755 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.769 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.779 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.790 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.308.802 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=1. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.308.983 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=1.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.309.017 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.047 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.061 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.073 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.084 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.094 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=2. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.309.376 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=2.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.309.413 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.431 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.445 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.460 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.474 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.309.486 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=3. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.310.093 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=3.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.310.127 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.145 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.159 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.174 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.188 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.200 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=4. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.310.632 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=4.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.310.666 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.681 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.694 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.706 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.718 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.310.729 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=5. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.311.796 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=5.) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.311.839 [memory.cpp:559]619761 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.311.855 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.311.869 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.311.882 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.311.896 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.311.911 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=7. [INFO] DRV(619443,python3.9):2025-04-03-09:35:45.312.840 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=7.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.312.885 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.312.946 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.312.980 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.020 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.061 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.081 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.103 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.121 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.143 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.177 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.197 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.213 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.240 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.257 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.275 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.292 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.312 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.347 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.375 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.393 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.422 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.438 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.456 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.474 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.498 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.533 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.563 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.583 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.605 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.620 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.640 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.658 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.675 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.689 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.712 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.729 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.744 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.758 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.784 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.801 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.819 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.850 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.870 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.885 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.901 [runtime.cc:1664] 619761 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.915 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.936 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.960 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.313.979 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.010 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.036 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.048 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.061 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.079 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.099 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.121 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.134 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.147 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.162 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.178 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.206 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.220 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.233 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.247 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.262 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.287 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.300 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.314 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.330 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.345 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.376 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.388 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.403 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.423 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.440 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.468 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.483 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.494 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.508 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.523 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.544 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.556 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.314.571 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.314 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.365 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.410 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.424 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.439 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.455 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.475 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.514 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.528 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.541 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.555 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.576 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.608 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.622 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.636 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.652 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.672 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.718 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.734 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.749 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.765 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.781 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.820 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.835 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.848 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.865 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.885 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.925 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.939 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.954 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.970 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.315.987 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.027 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.045 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.060 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.074 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.095 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.138 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.154 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.223 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.243 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.260 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.297 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.314 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.338 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.354 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.374 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.412 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.426 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.441 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.458 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.473 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.510 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.525 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.540 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.555 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.576 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.613 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.625 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.638 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.653 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.671 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.708 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.720 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.734 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.749 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.768 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.804 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.818 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.831 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.855 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.873 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.909 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.922 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.938 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.951 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.316.973 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.007 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.023 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.037 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.051 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.067 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.108 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.122 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.137 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.153 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.168 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.202 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.217 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.229 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.245 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.260 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.295 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.312 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.324 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.339 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.356 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.402 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.414 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.430 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.445 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.466 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.502 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.516 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.530 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.546 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.564 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.599 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.613 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.626 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.671 [api_impl.cc:1990] 619761 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.317.997 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.318.031 [api_impl.cc:2664] 619761 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.395 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619419, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.425 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619423, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.446 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619427, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.465 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619431, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.480 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619435, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.497 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619439, num=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.512 [npu_driver.cc:3571] 619761 SetIpcMemPid: set ipc mem pid success, name=000973b300000000000003190600ffd80006, pid[0]=619447, num=1 [2025-04-03 09:35:44.740874] [debug] [619443] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.740887] [debug] [619443] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.258219] [warn] [619761] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.258240] [debug] [619761] [lcal_sock_exchange.cpp:140] curRank: 6 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.258247] [debug] [619761] [lcal_sock_exchange.cpp:272] Client side 6 begin to connect [2025-04-03 09:35:45.258398] [debug] [619761] [lcal_sock_exchange.cpp:93] rank:6 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308320] [debug] [619761] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 6 rank :6 rankSize :8 [2025-04-03 09:35:45.308515] [debug] [619761] [lcal_comm.cpp:406] rank 6 devId: 6, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308528] [info] [619761] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308534] [info] [619761] [lcal_comm.cpp:242] rank 6/8 running devId:6 [2025-04-03 09:35:45.308583] [debug] [619761] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308610] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308731] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309009] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309402] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310118] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310657] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311830] [debug] [619761] [lcal_comm.cpp:340] 6 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312863] [debug] [619761] [lcal_comm.cpp:372] EnablePeerAccess succeed6 [2025-04-03 09:35:45.317640] [debug] [619761] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 6 [2025-04-03 09:35:45.317658] [debug] [619761] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.318018] [debug] [619761] [lcal_comm.cpp:455] peerMem[rank6], allocate finished. [2025-04-03 09:35:45.340276] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340293] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340303] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340312] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340318] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340326] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340331] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340338] [debug] [619761] [lcal_comm.cpp:472] rank : 6, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340345] [debug] [619761] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340527] [debug] [619761] [lcal_comm.cpp:557] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340823] [debug] [619761] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340834] [debug] [619761] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340842] [debug] [619761] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340849] [debug] [619761] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340872] [debug] [619761] [lcal_comm.cpp:515] rank[INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.340.931 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.342.008 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.342.031 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.342.876 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.342.893 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.343.373 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.343.390 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.343.795 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.343.811 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.357.976 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.358.000 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.108 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.124 [api_impl.cc:4286] 619761 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.673 [npu_driver.cc:3195] 619761 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.712 [api_impl.cc:1990] 619761 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.736 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.359.753 [memory.cpp:421]619761 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:45.359.897 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.359.930 [device.cpp:143]619761 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.953 [api_impl.cc:2999] 619761 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.967 [device.cc:306] 619761 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.982 [runtime.cc:3393] 619761 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.359.996 [runtime.cc:3560] 619761 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.010 [api_impl.cc:2987] 619761 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.022 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.036 [api_impl.cc:2992] 619761 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.051 [api_impl.cc:3025] 619761 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.360.075 [acl.cpp:115]619761 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.360.088 [acl.cpp:121]619761 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.360.103 [device.cpp:154]619761 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [WARNING] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:45.360.136 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.218 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.237 [adapter_rts.cc:198] [619761][hrtGetDeviceRefresh]deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.254 [externalinput.cc:2065] [619761]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.276 [externalinput.cc:490] [619761]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.293 [externalinput.cc:429] [619761]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.311 [externalinput.cc:543] [619761]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.328 [externalinput.cc:622] [619761]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.344 [externalinput.cc:714] [619761]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.361 [externalinput.cc:821] [619761]HCCL_IF_IP is not set [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.377 [externalinput.cc:877] [619761]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.391 [externalinput.cc:844] [619761]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.403 [externalinput.cc:728] [619761]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.418 [externalinput.cc:758] [619761]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.430 [externalinput.cc:806] [619761]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.443 [externalinput.cc:1508] [619761]HCCL_ALGO is not set [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.457 [externalinput.cc:1671] [619761][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.473 [externalinput.cc:462] [619761]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.486 [externalinput.cc:1519] [619761]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.499 [externalinput.cc:1554] [619761]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.360.594 [api_c_soc.cc:70] 619761 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.613 [externalinput.cc:1599] [619761]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.634 [externalinput.cc:1633] [619761]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.648 [externalinput.cc:1778] [619761]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.663 [externalinput.cc:1807] [619761]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.689 [externalinput.cc:515] [619761]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.705 [externalinput.cc:1841] [619761]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.718 [externalinput.cc:1929] [619761]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.731 [externalinput.cc:1949] [619761]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.744 [externalinput.cc:1969] [619761]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.766 [externalinput.cc:2005] [619761]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.782 [externalinput.cc:2036] [619761]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.799 [externalinput.cc:2209] [619761]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.814 [externalinput.cc:2178] [619761][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.827 [externalinput.cc:889] [619761]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.840 [externalinput.cc:914] [619761]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.854 [externalinput.cc:2240] [619761]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.868 [externalinput.cc:2308] [619761]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.884 [externalinput.cc:2339] [619761]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.898 [externalinput.cc:2296] [619761]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.360.913 [externalinput.cc:407] [619761]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:45.360.931 [externalinput.cc:672] [619761][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:45.360.978 [env_config.cc:127] [619761]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:45.360.996 [env_config.cc:131] [619761]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.012 [env_config.cc:302] [619761]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.023 [env_config.cc:302] [619761]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.035 [op_base.cc:1091] [619761]Entry-HcclCommInitRootInfoInner:ranks[8], rank[6], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.074 [op_base.cc:128] [619761][HcclGetDeviceId] deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.107 [op_base.cc:144] [619761][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.146 [rank_consistentcy_checker.cc:32] [619761][GetInstance] get deviceLogicId[6] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.361.178 [api_impl.cc:3100] 619761 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.361.191 [api_impl.cc:3111] 619761 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.361.217 [api_impl.cc:3130] 619761 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.439 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.459 [network_manager.cc:169] [619761]NetworkManager: devicePhyId[6], devicePhyId_[6] deviceLogicId_[6], hasBackup[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.361.492 [ra_host.c:1594]tid:619761,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.361.515 [adapter_hccp.cc:1450] [619761]set host socket whitelist status[0] success. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.361.554 [ra_host.c:205]tid:619761,ra_init(205) : Input parameters: phy_id[6], nic_position:[0] hdc_type:[6] [INFO] ROCE(619443,python3.9):2025-04-03-09:35:45.361.613 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.361.631 [ra_peer.c:776]tid:619761,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[6] start [INFO] ROCE(619443,python3.9):2025-04-03-09:35:45.361.696 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619443,python3.9):2025-04-03-09:35:45.361.764 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.361.793 [rs.c:327]tid:619761,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619443,python3.9):2025-04-03-09:35:45.361.951 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619443,python3.9):2025-04-03-09:35:45.361.976 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619443,python3.9):2025-04-03-09:35:45.361.991 [rs_ssl.c:1513]tid:619761,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.005 [rs_ssl.c:1529]tid:619761,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.038 [rs_epoll.c:692]tid:619761,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.203 [rs_epoll.c:487]tid:619765,rs_epoll_handle(487) : thread begin! thread_id:281468936908704, pid:619443, ppid:619411 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.238 [rs_epoll.c:713]tid:619761,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.250 [rs_epoll.c:415]tid:619765,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.247 [rs_epoll.c:623]tid:619767,rs_connect_handle(623) : thread begin! thread_id:281468928454560, pid:619443, ppid:619411 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.256 [rs.c:366]tid:619761,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.237000] ms [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.267 [rs_epoll.c:511]tid:619765,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.362.318 [rs_epoll.c:641]tid:619767,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.199 [rs.c:471]tid:619761,rs_init(471) : rs init success, chip_id[6] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.226 [ra_peer.c:813]tid:619761,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[6] succ [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.370.244 [adapter_hccp.cc:583] [619761]init ra success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.370.259 [network_manager.cc:222] [619761]NetworkManager nicDeploy[0] deviceLogicId_[6] devicePhyId_[6] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.301 [ra_host.c:1764]tid:619761,ra_get_ifnum(1764) : Input parameters: phy_id[6], nic_position:[0], is_all:[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.321 [ra_peer.c:859]tid:619761,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[6] start [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.342 [ra_peer.c:867]tid:619761,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[6] succ [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.370.356 [adapter_hccp.cc:1378] [619761][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.374 [ra_host.c:1814]tid:619761,ra_get_ifaddrs(1814) : Input parameters: phy_id[6], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.389 [ra_peer.c:874]tid:619761,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[6] start [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.403 [rs_socket.c:2743]tid:619761,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.418 [rs_socket.c:2743]tid:619761,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.432 [rs_socket.c:2743]tid:619761,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.446 [rs_socket.c:2749]tid:619761,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.460 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.471 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.483 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.494 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.504 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.514 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.527 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.540 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.555 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.569 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.582 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.594 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.606 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.616 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.629 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.641 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.652 [rs_socket.c:2749]tid:619761,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.663 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.676 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.696 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.708 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.720 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.731 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.742 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.752 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.763 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.775 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.786 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.799 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.809 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.820 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.830 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.843 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.857 [rs_socket.c:2749]tid:619761,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.871 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.881 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.891 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.903 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.915 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.928 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.939 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.948 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.959 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.972 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.983 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.370.994 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.005 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.017 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.029 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.047 [rs_socket.c:2751]tid:619761,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.061 [ra_peer.c:882]tid:619761,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[6] succ [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.083 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.099 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.113 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.127 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.144 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.157 [adapter_hccp.cc:1407] [619761][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.183 [sal.cc:365] [619761]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.199 [sal.cc:432] [619761]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.217 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:45.371.249 [ra_host.c:340]tid:619761,ra_socket_init_v1(340) : socket init:mode=0 phy_id=6 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.263 [adapter_hccp.cc:818] [619761]socket init v1 success, socketHandle[0xfffe900faba0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.280 [network_manager.cc:1218] [619761]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.291 [network_manager.cc:1257] [619761]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe900faba0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.346 [adapter_hal.cc:298] [619761]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.370 [sal.cc:593] [619761]IsHostUseDevNic[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.386 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.400 [network_manager.cc:169] [619761]NetworkManager: devicePhyId[4294967295], devicePhyId_[6] deviceLogicId_[6], hasBackup[0] [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.442 [tsd_client.cpp:189][TsdCapabilityGet][tid:619761] TsdCapabilityGet Begin. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.456 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.470 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.481 [process_mode_manager.cpp:1042][CapabilityGet][tid:619761] [ProcessModeManager] enter into CapabilityGet process deviceId[6] type[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.494 [adapter_tdt.cc:66] [619761]Get TsdCapability success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.506 [network_manager.cc:175] [619761][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:45.371.527 [network_manager.cc:99] [619761][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.544 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.564 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.576 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619761] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.590 [process_mode_manager.cpp:386][InitTsdClient][tid:619761] [TsdClient] tsd client has been inited before [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.608 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619761] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619443,python3.9):2025-04-03-09:35:45.371.629 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619761] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.643 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619761] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.659 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619761] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.674 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619761] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.696 [version_verify.cpp:114][SpecialFeatureCheck][tid:619761] VersionVerify: previous type[48], supported [INFO] TDT(619443,python3.9):2025-04-03-09:35:45.371.742 [hdc_client.cpp:154][TsdRecvData][tid:619761] recv timeout is:150000 ms [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.380.174 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.380.192 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.380.201 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.380.218 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.226 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.236 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.242 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.253 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.380.259 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.380.270 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.380.285 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.380.294 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.300 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.380.312 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.166 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.171 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.173 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.176 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.177 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.572.176 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.281 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.302 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.321 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.336 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.345 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.349 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.362 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.367 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.379 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.382 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.394 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.402 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.409 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.416 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.419 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.445 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.450 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.455 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.459 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.472 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.479 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.488 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.493 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.502 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.507 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.518 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.522 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.524 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.530 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.554 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.559 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.575 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.639 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.572.655 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.687 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.572.716 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.892.179 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.892.179 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.892.209 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:45.892.238 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.242 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.253 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.258 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.267 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.892.272 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.892.280 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.892.286 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:45.892.294 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.301 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:45.892.312 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.252.702 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:46.252.724 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:46.252.738 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:46.252.753 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.166 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.166 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.166 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.172 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.177 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.187 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.199 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.212 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.243 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.404.267 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.287 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.292 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.292 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.302 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.304 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.312 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.316 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.322 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.322 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.332 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.337 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.347 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.350 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.352 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.373 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.364 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.415 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.383 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.384 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.392 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.397 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.404 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.460 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.419 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.424 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.440 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.450 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.463 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.468 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.491 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.501 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.512 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.518 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.523 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.527 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.537 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.575 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.582 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.598 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.546 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.404.563 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.588 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.612 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.622 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.632 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.404.666 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.916.172 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.916.174 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.916.200 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:46.916.225 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.235 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.241 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.249 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.257 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.916.263 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.916.270 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.916.277 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:46.916.286 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.294 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:46.916.303 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.169 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.170 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.172 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.175 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.174 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.236.176 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.288 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.299 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.304 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.314 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.319 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.322 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.329 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.351 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.356 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.382 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.374 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.400 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.364 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.386 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.391 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.406 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.424 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.423 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.435 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.444 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.449 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.459 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.473 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.483 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.493 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.511 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.522 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.530 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.541 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.584 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.555 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.236.565 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.575 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.660 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.667 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.236.676 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.428.175 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.428.176 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.428.202 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.428.210 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.221 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.228 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.236 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.242 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.428.251 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.428.259 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.428.265 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.428.272 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.281 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.428.286 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.730.964 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.730.969 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619761] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.731.002 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619761] [TsdClient] DeviceMsgProc recvMsg realDeviceId[6] msgType[49] localDevId[6] rspCode[0]heterogeneousSubPid[30805], tsdSupportLevel_[63] [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.731.021 [hdc_client.cpp:172][TsdRecvData][tid:619761] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.731.036 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619761] OpenSubProc success stype:0, pid:30805 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.731.045 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.731.051 [adapter_tdt.cc:39] [619761]Open TsdProcess success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.731.090 [network_manager.cc:118] [619761][TsdProcessOpen]hrtTsdProcessOpen success, subPid[30805], devicePhyId_[6], deviceLogicId_[6], hasBackup[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.731.106 [network_manager.cc:180] [619761][NetworkManager][Init]open tsd by process success, devicePhyId[6], deviceLogicId_[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.731.147 [ra_host.c:205]tid:619761,ra_init(205) : Input parameters: phy_id[6], nic_position:[1] hdc_type:[18] [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.731.165 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619443,python3.9):2025-04-03-09:35:47.731.186 [devdrv_hccl.c:103][ascend][curpid:619443,619761][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619443) [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.731.204 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.731.223 [ra_hdc.c:580]tid:619761,ra_hdc_init(580) : hdc init start! logic id is 6, phy id is 6, hdc_type is 18 [INFO] DRV(619443,python3.9):2025-04-03-09:35:47.731.433 [hdc_core.c:1982][ascend][curpid:619443,619761][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4264; pid=619443) [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.731.665 [ra_hdc.c:616]tid:619761,ra_hdc_init(616) : hdc init OK! phy_id[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.735.457 [adapter_hccp.cc:583] [619761]init ra success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.735.476 [network_manager.cc:222] [619761]NetworkManager nicDeploy[1] deviceLogicId_[6] devicePhyId_[6] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.735.501 [topoinfo_detect.cc:534] [619761]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.735.521 [ra_host.c:1764]tid:619761,ra_get_ifnum(1764) : Input parameters: phy_id[6], nic_position:[1], is_all:[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.735.901 [adapter_hccp.cc:1538] [619761][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.735.920 [ra_host.c:1814]tid:619761,ra_get_ifaddrs(1814) : Input parameters: phy_id[6], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.262 [adapter_hccp.cc:1568] [619761][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth6] ip[29.105.134.38%eth6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.289 [adapter_hccp.cc:1568] [619761][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth6] ip[fe80::2287:ecff:fe02:45f5%eth6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.307 [topoinfo_detect.cc:577] [619761]select AF_INET family as device socket family. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.325 [topoinfo_detect.cc:756] [619761]no backup device ip: use 0 as device ip. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.340 [topoinfo_detect.cc:294] [619761][HCCL_TRACE]SetupAgent rankNum[8], rank[6], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[6], phydevId[6], deviceIp[29.105.134.38] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.372 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.391 [hccl_socket.cc:516] [619761][Get][NicHandleInfo]phyId[6], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.407 [hccl_socket.cc:217] [619761][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281468803722144], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.736.435 [adapter_hccp.cc:1050] [619761]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.736.453 [ra_host.c:759]tid:619761,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[6], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.736.483 [rs_socket.c:1810]tid:619761,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.736.502 [rs_socket.c:1819]tid:619761,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619443,python3.9):2025-04-03-09:35:47.736.518 [rs_socket.c:132]tid:619761,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.736.537 [rs_socket.c:1880]tid:619761,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.737.199 [rs_socket.c:1577]tid:619767,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:57845, fd:53 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.737.231 [rs_drv_socket.c:59]tid:619767,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:53 begin [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.737.311 [rs_drv_socket.c:98]tid:619767,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:57845, server addr 192.168.0.44:60008, fd:53 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.737.334 [rs_socket.c:1427]tid:619767,rs_conn_cost_time(1427) : socket [53] connect success! cost [0.797000] ms [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.737.376 [rs_socket.c:1392]tid:619767,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.737.624 [topoinfo_exchange_agent.cc:212] [619761]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.737.678 [topoinfo_exchange_agent.cc:236] [619761]local rank[6] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.737.696 [topoinfo_exchange_agent.cc:43] [619761]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.737.718 [topoinfo_exchange_agent.cc:345] [619761]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.737.814 [topoinfo_exchange_agent.cc:77] [619761]topo exchange client send rank basic info success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.801.019 [api_impl.cc:2096] 619761 HostMalloc: size=2485 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.801.062 [api_error.cc:1260] 619761 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.801.338 [api_error.cc:1278] 619761 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.359 [topoinfo_exchange_agent.cc:80] [619761]topo exchange client get rank basic info success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.410 [topoinfo_detect.cc:303] [619761]topo detect completed. myrank[6], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.442 [config.cc:198] [619761]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.479 [config.cc:210] [619761]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.497 [config.cc:210] [619761]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.516 [config.cc:210] [619761]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.530 [config.cc:210] [619761]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.547 [config.cc:210] [619761]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.563 [config.cc:210] [619761]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.576 [config.cc:210] [619761]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.592 [config.cc:210] [619761]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.801.609 [adapter_hccp.cc:1067] [619761]ra socket batch close fdhandle[0xfffe90054bf0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.801.627 [ra_host.c:796]tid:619761,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[6], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.801.652 [rs_socket.c:1937]tid:619761,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:53, state:9 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.848.190 [adapter_hccp.cc:1099] [619761]ra socket batch close success,take time [46565]us [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:47.848.211 [hccl_socket.cc:232] [619761][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.848.231 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.251 [ra_host.c:404]tid:619761,ra_socket_deinit(404) : Input parameters: phy_id[6] family[2] local_ip[192.168.0.44] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.267 [rs.c:1629]tid:619761,rs_socket_deinit(1629) : rs socket deinit start, phy_id:6 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.282 [rs.c:1641]tid:619761,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.296 [rs.c:1653]tid:619761,rs_socket_deinit(1653) : socket deinit success, phy_id:6, local_ip:192.168.0.44 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.848.310 [network_manager.cc:1233] [619761]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.848.323 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.338 [ra_host.c:248]tid:619761,ra_deinit(248) : Input parameters: phy_id[6], nic_position:[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.353 [ra_peer.c:821]tid:619761,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[6] start [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.373 [rs.c:1783]tid:619761,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619443,python3.9):2025-04-03-09:35:47.848.378 [rs_epoll.c:523]tid:619765,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:51 [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.848.419 [rs_epoll.c:544]tid:619765,rs_epoll_handle(544) : QUIT thread_id:281468936908704, pid:619443, read num:8 [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.940.178 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.940.178 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.940.201 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:47.940.211 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.217 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.229 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.235 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.244 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.940.250 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.940.260 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.940.264 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:47.940.275 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.280 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:47.940.293 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.947.820 [rs_epoll.c:679]tid:619767,rs_connect_handle(679) : QUIT thread_id:281468928454560, pid:619443 [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.948.669 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.948.688 [rs.c:1810]tid:619761,rs_deinit(1810) : rs_deinit chip_id[6] ok [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.948.703 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.948.720 [ra_peer.c:852]tid:619761,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[6] succ [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.948.735 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.948.761 [network_manager.cc:409] [619761]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.948.779 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.948.794 [ra_host.c:248]tid:619761,ra_deinit(248) : Input parameters: phy_id[6], nic_position:[1] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.948.806 [ra_hdc.c:651]tid:619761,ra_hdc_deinit(651) : hdc deinit start! phy_id[6] [INFO] DRV(619443,python3.9):2025-04-03-09:35:47.948.951 [hdc_client.c:417][ascend][curpid:619443,619761][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4264) [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.948.999 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:47.949.014 [ra_hdc.c:685]tid:619761,ra_hdc_deinit(685) : hdc deinit OK! phy_id[6] [INFO] ROCE(619443,python3.9):2025-04-03-09:35:47.949.027 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.051 [tsd_client.cpp:189][TsdCapabilityGet][tid:619761] TsdCapabilityGet Begin. [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.070 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.084 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.098 [process_mode_manager.cpp:1042][CapabilityGet][tid:619761] [ProcessModeManager] enter into CapabilityGet process deviceId[6] type[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:47.949.112 [adapter_tdt.cc:66] [619761]Get TsdCapability success. deviceLogicId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.124 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.136 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.149 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619761] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.163 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619761] add close subproc:30805, proctype:0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.178 [version_verify.cpp:114][SpecialFeatureCheck][tid:619761] VersionVerify: previous type[60], supported [INFO] TDT(619443,python3.9):2025-04-03-09:35:47.949.204 [hdc_client.cpp:154][TsdRecvData][tid:619761] recv timeout is:150000 ms [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.168 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.172 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.173 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.182 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.184 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.068.191 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.297 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.306 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.314 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.338 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.339 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.349 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.369 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.399 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.400 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.411 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.418 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.427 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.434 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.438 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.446 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.457 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.462 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.469 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.477 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.520 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.488 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.496 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.507 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.514 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.528 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.533 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.543 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.549 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.559 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.068.570 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.575 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.586 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.615 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.634 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.650 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.068.666 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.201.385 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619761] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.201.413 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619761] [TsdClient] DeviceMsgProc recvMsg realDeviceId[6] msgType[61] localDevId[6] rspCode[0]heterogeneousSubPid[30805], tsdSupportLevel_[63] [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.201.431 [hdc_client.cpp:172][TsdRecvData][tid:619761] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.201.450 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619761] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.201.463 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619761] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.477 [adapter_tdt.cc:52] [619761]Open TsdProcess success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.490 [network_manager.cc:405] [619761][DeInit]finish CloseHccpProcess, phy_id[6], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.506 [network_manager.cc:409] [619761]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.521 [topoinfo_detect.cc:555] [619761]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.534 [topoinfo_detect.cc:321] [619761]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.548 [topoinfo_detect.cc:219] [619761]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.562 [rank_consistentcy_checker.cc:32] [619761][GetInstance] get deviceLogicId[6] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:48.201.578 [rank_consistentcy_checker.cc:144] [619761][Record][CannVersion] version information str is empty. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.590 [op_base.cc:2231] [619761]rank table is null, rankTableCrc is 0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.630 [op_base.cc:2219] [619761]params.id.internal [hccl-619443-0-1743644148-hccl_world_group] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.647 [op_base.cc:993] [619761]rootInfo[hccl-619443-0-1743644148-hccl_world_group], params.logiceDevice[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.660 [hccl_comm.cc:98] [619761]hcclComm init workmode [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.201.673 [hccl_comm.cc:954] [619761]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.202.592 [hccl_comm.cc:1365] [619761]RegistTaskAbortHandler begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.202.612 [task_abort_handler.cc:126] [619761]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.202.641 [api_impl.cc:4192] 619761 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.202.659 [runtime.cc:4698] 619761 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.202.700 [adapter_rts.cc:42] [619761]open so: libruntime.so [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.202.951 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.202.992 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.013 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.036 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.053 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.072 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.087 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.107 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.123 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.142 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.157 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.178 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.192 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.213 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.229 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.249 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.265 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.284 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.297 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.318 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.333 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.353 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.366 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.397 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.412 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.430 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.444 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.463 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.478 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.495 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.508 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.526 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.538 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.559 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.575 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.596 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.608 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.629 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.643 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.661 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.675 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.694 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.707 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.722 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.735 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.752 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.764 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.788 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.803 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.821 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.835 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.853 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.865 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.883 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.203.897 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.917 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.964 [hccl_communicator_attrs.cc:407] [619761]module[0] contains [8]devices [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.979 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.203.991 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.004 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.016 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.031 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.044 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.057 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.070 [hccl_communicator_attrs.cc:411] [619761]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.083 [hccl_communicator_attrs.cc:425] [619761]different module contains different numbers of cards:[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.094 [hccl_communicator_attrs.cc:426] [619761]different module contains different type of cards:[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.145 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.185 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.209 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.231 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.249 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.278 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.292 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.313 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.329 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.348 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.362 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.381 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.395 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.412 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.424 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.444 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.460 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.481 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.495 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.516 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.530 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.551 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.567 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.589 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.601 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.624 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.638 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.656 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.669 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.697 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.712 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.733 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.747 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.770 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.785 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.804 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.818 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.839 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.852 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.870 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.885 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.907 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.924 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.945 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.959 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.204.977 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.204.991 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.012 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.026 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.046 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.060 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.084 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.098 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.116 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.145 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.161 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.174 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.192 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.207 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.224 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.236 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.255 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.268 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.287 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.303 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.322 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.335 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.356 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.371 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.388 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.401 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.424 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.439 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.457 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.473 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.491 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.504 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.522 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.537 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.563 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.576 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.596 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.611 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.629 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.644 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.663 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.676 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.697 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.711 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.729 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.741 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.761 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.774 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.793 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.806 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.827 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.842 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.859 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.871 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.892 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.905 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.921 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.934 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.205.953 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.205.976 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.003 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.018 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.043 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.057 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.075 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.088 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.108 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.130 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.150 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.163 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.181 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.193 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.214 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.226 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.249 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.262 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.283 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.296 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.315 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.330 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.346 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.358 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.374 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.396 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.417 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.430 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.448 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.462 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.481 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.492 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.511 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.523 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.543 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.557 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.576 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.589 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.609 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.622 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.639 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.650 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.667 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.680 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.697 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.709 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.725 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.739 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.756 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.767 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.796 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.809 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.828 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.842 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.861 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.874 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.899 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.914 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.934 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.947 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.966 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.206.977 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.206.996 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.009 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.027 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.067 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.089 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.108 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.124 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.140 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.159 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.183 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.200 [hccl_communicator_attrs.cc:156] [619761][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.223 [hccl_communicator_attrs.cc:1021] [619761][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.239 [hccl_communicator.cc:1440] [619761][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.252 [hccl_communicator.cc:1450] [619761][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.268 [externalinput.cc:1445] [619761]InitParsesProtocol start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.283 [externalinput.cc:1462] [619761]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:48.207.298 [externalinput.cc:1411] [619761]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.321 [externalinput.cc:1413] [619761]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.334 [externalinput.cc:1467] [619761]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.371 [config.cc:557] [619761][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.388 [hccl_communicator.cc:668] [619761][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.412 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.434 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.449 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=0. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.468 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=0.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.482 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.497 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.514 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.529 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=1. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.545 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=1.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.557 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.573 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.588 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.608 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=2. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.624 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=2.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.639 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.655 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.669 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.682 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=3. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.698 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=3.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.711 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.724 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.738 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.751 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=4. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.768 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=4.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.782 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.793 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.811 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.828 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=5. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.847 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=5.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.859 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.872 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.887 [api_impl.cc:3520] 619761 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.207.902 [api_impl.cc:3168] 619761 EnableP2P: Enable P2P devIdDes=6, phyIdSrc=7. [INFO] DRV(619443,python3.9):2025-04-03-09:35:48.207.918 [devdrv_pcie.c:888][ascend][curpid:619443,619761][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=6; peer_phy_devid=7.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.931 [p2p_mgmt.cc:74] [619761]enable p2p: local logic id:6, remote physic id:7. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.955 [adapter_hal.cc:298] [619761]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.207.976 [sal.cc:593] [619761]IsHostUseDevNic[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.023 [adapter_hal.cc:298] [619761]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.043 [sal.cc:593] [619761]IsHostUseDevNic[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.056 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.067 [network_manager.cc:169] [619761]NetworkManager: devicePhyId[4294967295], devicePhyId_[6] deviceLogicId_[6], hasBackup[0] [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.082 [tsd_client.cpp:189][TsdCapabilityGet][tid:619761] TsdCapabilityGet Begin. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.105 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.120 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.131 [process_mode_manager.cpp:1042][CapabilityGet][tid:619761] [ProcessModeManager] enter into CapabilityGet process deviceId[6] type[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.143 [adapter_tdt.cc:66] [619761]Get TsdCapability success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.154 [network_manager.cc:175] [619761][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:48.208.189 [network_manager.cc:99] [619761][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.207 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.217 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619761] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.228 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619761] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.239 [process_mode_manager.cpp:386][InitTsdClient][tid:619761] [TsdClient] tsd client has been inited before [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.255 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619761] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619443,python3.9):2025-04-03-09:35:48.208.269 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619761] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.281 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619761] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.293 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619761] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.310 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619761] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.208.350 [hdc_client.cpp:154][TsdRecvData][tid:619761] recv timeout is:150000 ms [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.452.172 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.452.177 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.452.199 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.452.210 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.231 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.238 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.245 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.251 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.452.259 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.452.266 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.452.274 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.452.281 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.291 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.452.296 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.167 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.169 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.171 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.172 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.179 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.900.181 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.267 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.285 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.298 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.312 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.324 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.329 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.337 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.343 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.364 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.365 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.377 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.389 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.398 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.402 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.416 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.408 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.424 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.440 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.449 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.440 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.458 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.467 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.478 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.487 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.482 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.498 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.507 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.532 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.537 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.539 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.564 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.577 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.567 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.900.623 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.664 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.900.691 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.940.713 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.940.746 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.940.764 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:48.940.779 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.964.174 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.964.174 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.964.197 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:48.964.205 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.213 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.224 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.232 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.240 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.964.246 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.964.253 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.964.259 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:48.964.266 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.274 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:48.964.281 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.476.166 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.476.167 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.476.192 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.476.199 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.223 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.230 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.237 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.243 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.476.251 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.476.257 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.476.265 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.476.271 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.282 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.476.289 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.169 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.171 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.176 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.181 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.181 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.732.185 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.262 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.279 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.293 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.304 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.308 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.316 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.321 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.338 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.346 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.354 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.389 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.375 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.379 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.402 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.408 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.413 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.436 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.443 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.451 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.420 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.430 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.449 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.456 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.470 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.476 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.492 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.496 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.508 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.543 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.732.568 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.552 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.559 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.581 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.574 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.617 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.732.628 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.988.175 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.988.179 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.988.201 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:49.988.211 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.216 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.227 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.233 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.241 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.988.249 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.988.256 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.988.263 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:49.988.272 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.280 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:49.988.287 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] TDT(619443,python3.9):2025-04-03-09:35:50.309.532 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619761] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.309.536 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:50.309.556 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619761] [TsdClient] DeviceMsgProc recvMsg realDeviceId[6] msgType[49] localDevId[6] rspCode[0]heterogeneousSubPid[30824], tsdSupportLevel_[63] [INFO] TDT(619443,python3.9):2025-04-03-09:35:50.309.572 [hdc_client.cpp:172][TsdRecvData][tid:619761] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:50.309.588 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619761] OpenSubProc success stype:0, pid:30824 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.309.603 [adapter_tdt.cc:39] [619761]Open TsdProcess success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.309.616 [network_manager.cc:118] [619761][TsdProcessOpen]hrtTsdProcessOpen success, subPid[30824], devicePhyId_[6], deviceLogicId_[6], hasBackup[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.309.619 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.309.643 [network_manager.cc:180] [619761][NetworkManager][Init]open tsd by process success, devicePhyId[6], deviceLogicId_[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:50.309.693 [ra_host.c:205]tid:619761,ra_init(205) : Input parameters: phy_id[6], nic_position:[1] hdc_type:[18] [INFO] ROCE(619443,python3.9):2025-04-03-09:35:50.309.752 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619443,python3.9):2025-04-03-09:35:50.309.773 [devdrv_hccl.c:103][ascend][curpid:619443,619761][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619443) [INFO] ROCE(619443,python3.9):2025-04-03-09:35:50.309.810 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:50.309.826 [ra_hdc.c:580]tid:619761,ra_hdc_init(580) : hdc init start! logic id is 6, phy id is 6, hdc_type is 18 [INFO] DRV(619443,python3.9):2025-04-03-09:35:50.310.000 [hdc_core.c:1982][ascend][curpid:619443,619761][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4271; pid=619443) [INFO] HCCP(619443,python3.9):2025-04-03-09:35:50.310.301 [ra_hdc.c:616]tid:619761,ra_hdc_init(616) : hdc init OK! phy_id[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.050 [adapter_hccp.cc:583] [619761]init ra success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.068 [network_manager.cc:222] [619761]NetworkManager nicDeploy[1] deviceLogicId_[6] devicePhyId_[6] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.086 [hccl_communicator_attrs.cc:1307] [619761][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.099 [hccl_communicator_attrs.cc:1275] [619761]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.117 [hccl_communicator.cc:2259] [619761][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.142 [adapter_hccp.cc:1650] [619761]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.324 [adapter_hccp.cc:2343] [619761]Get available Vnic info success, deviceIdType[0] deviceId[0x6] Vnic ip[192.168.2.193] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.365 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:50.314.383 [ra_host.c:280]tid:619761,ra_socket_init(280) : socket init:mode=1 phy_id=6 family=2 ip=192.168.2.193 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.400 [adapter_hccp.cc:806] [619761]socket init success, ip[3238176960] device id[6], socketHandle[0xfffe900ffbb0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.417 [network_manager.cc:692] [619761]devicePhyId[6], ip[192.168.2.193] socket init OK [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.429 [network_manager.cc:432] [619761][Start][Vnic]trying to listen on ip[192.168.2.193] port[16666]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:50.314.444 [ra_host.c:875]tid:619761,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[6], local_ip[192.168.2.193], port[16666] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.753 [network_manager.cc:448] [619761][Start][Vnic]Listen on ip[192.168.2.193], port[16666] sucess, devPhyId[6], devLogicId[6], isAutoPort[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.772 [network_manager.cc:451] [619761]NetworkManager devicePhyId_[6] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.785 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.815 [hccl_socket.cc:106] [619761][HcclSocket][Listen] device[6] listens on ip[192.168.2.193] port[16666] success, socketType[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.834 [hccl_socket_manager.cc:56] [619761][Init][Server]ip[192.168.2.193] port[16666] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.851 [hccl_communicator.cc:1548] [619761][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.193] port[16666] success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.867 [hccl_communicator_attrs.cc:1307] [619761][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.878 [hccl_communicator_attrs.cc:1275] [619761]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.896 [hccl_communicator.cc:1565] [619761]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.908 [network_manager.cc:69] [619761]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.314.922 [hccl_communicator.cc:2028] [619761]profiling config information:options[], mode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.032 [adapter_trace.cc:21] [619761]Call TraceCreate, return value[0] [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.315.055 [atrace_api.c:61](tid:619761) AtraceCreateWithAttr start. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.315.131 [trace_rb_log.c:45](tid:619761) init ring buffer time finished, realTime 1743644150315124404, monotonicTime 4211952505701, freq 99993 kHz, minutesWest 480 minutes [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.315.150 [trace_rb_log.c:137](tid:619761) [HCCL_619761_6] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.315.170 [tracer_mgr_operate.c:66](tid:619761) create object HCCL_619761_6 success, exitSave(true), noLock(false). [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.315.186 [atrace_api.c:63](tid:619761) AtraceCreateWithAttr end. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.199 [adapter_trace.cc:70] [619761]Call TraceCreateWithAttr, return value[0xaaaaf40448f8], Params: objName[HCCL_619761_6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.231 [rank_consistentcy_checker.cc:32] [619761][GetInstance] get deviceLogicId[6] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.270 [api_impl.cc:1486] 619761 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.298 [npu_driver.cc:4604] 619761 GetMaxStreamAndTask: deviceId=6, tsId=0, streamCount=1984. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.316 [task_exception_handler.cc:1212] [619761]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.337 [hccl_communicator_attrs.cc:1099] [619761][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.390 [api_impl.cc:4219] 619761 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.412 [command_handle.cc:25] [619761][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.428 [command_handle.cc:99] [619761][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.536 [profiler_manager_impl.cc:76] [619761][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.554 [hccl_communicator.cc:2193] [619761][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.315.571 [hccl_communicator.cc:803] [619761]Will use ffts mode. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.611 [api_impl.cc:3138] 619761 GetDeviceIndexByPhyId: get Index by PhyId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.628 [api_impl.cc:3149] 619761 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.641 [api_impl.cc:3157] 619761 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.315.653 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.316.661 [api_impl.cc:3476] 619761 GetDeviceInfo: get device info, deviceId=6 [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:50.316.712 [tbe_vector_reduce.cc:508] [619761]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.467 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.629 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.317.649 [dispatcher.cc:157] [619761][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.671 [api_impl.cc:3138] 619761 GetDeviceIndexByPhyId: get Index by PhyId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.687 [api_impl.cc:3149] 619761 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.699 [api_impl.cc:3157] 619761 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.712 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.317.732 [api_impl.cc:1990] 619761 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.322.486 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.322.551 [api_impl.cc:1990] 619761 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.322.577 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.322.608 [api_impl.cc:1990] 619761 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.322.626 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.656 [opexecounter.cc:71] [619761]alloc counter mem resource. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.692 [hccl_communicator_attrs.cc:1368] [619761][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.738 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.757 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.779 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.794 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.808 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.823 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.848 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.863 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.876 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.889 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.902 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.915 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.930 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.943 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.955 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.969 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.983 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.322.998 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.015 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.030 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.042 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.058 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.071 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.085 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.099 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.115 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.129 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.141 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.162 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.174 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.187 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.201 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.212 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.227 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.242 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.255 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.268 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.282 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.296 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.310 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.324 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.337 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.351 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.368 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.382 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.397 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.411 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.424 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.439 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.454 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.466 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.490 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.505 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.519 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.533 [alg_configurator.cc:68] [619761][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.547 [alg_configurator.cc:241] [619761][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.561 [alg_configurator.cc:134] [619761]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.582 [alg_configurator.cc:449] [619761][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.600 [alg_configurator.cc:42] [619761]alg configurator init success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.636 [topo_info_extractor.cc:66] [619761]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[6], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.661 [topo_info_extractor.cc:155] [619761][SetRankInfo]rankData_: userRank[6], devicePhyId[6], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.685 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.716 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.751 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.776 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.798 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.820 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.837 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.857 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.893 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.915 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.933 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.323.952 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.323.973 [api_error.cc:3478] 619761 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.002 [adapter_rts.cc:940] [619761][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.022 [topo_info_extractor.cc:348] [619761]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.176 [topo_info_extractor.cc:888] [619761]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[6], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.210 [topo_info_extractor.cc:585] [619761][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.232 [topo_info_extractor.cc:646] [619761][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.251 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.268 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.287 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.303 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.321 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.341 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.358 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/1; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.378 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.392 [topo_info_extractor.cc:689] [619761]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[6], userRankSize[8], plane size[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.405 [topo_info_extractor.cc:697] [619761][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.419 [topo_info_extractor.cc:734] [619761]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 6/6/0/; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.433 [topo_info_extractor.cc:739] [619761]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[6], userRankSize[8], plane size[1] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.449 [topo_info_extractor.cc:585] [619761][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.464 [topo_info_extractor.cc:646] [619761][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.481 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.518 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.538 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.557 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.576 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.594 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.613 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/1; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.631 [topo_info_extractor.cc:685] [619761]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.646 [topo_info_extractor.cc:689] [619761]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[6], userRankSize[8], plane size[8] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.686 [topo_info_extractor.cc:790] [619761]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId6/6; ] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.706 [topo_info_extractor.cc:519] [619761]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.324.832 [api_impl.cc:1990] 619761 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.324.862 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.324.915 [hccl_impl.cc:59] [619761]start hccl resources build:get deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.325.029 [hccl_impl.cc:177] [619761]hcclImpl init success. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:50.325.058 [hccl_aiv.cc:279] [619761][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.326.940 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.003 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.111 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.133 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.154 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.174 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.198 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.295 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.323 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.340 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.356 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.378 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.480 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.498 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.514 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.531 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.552 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.648 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.666 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.682 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.697 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.718 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.810 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.827 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.842 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.857 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.878 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.966 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.327.983 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.000 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.016 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.037 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.126 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.141 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.172 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.193 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.215 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.304 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.320 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.336 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.350 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.372 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.465 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.482 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.498 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.514 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.534 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.626 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.642 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.657 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.673 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.693 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.783 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.800 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.816 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.831 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.854 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.949 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.966 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.328.982 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.011 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.033 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.122 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.138 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.154 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.171 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.193 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.283 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.298 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.314 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.328 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.348 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.437 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.453 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.469 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.484 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.504 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.596 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.612 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.628 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.642 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.663 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.752 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.769 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.785 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.800 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.819 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.917 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.933 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.948 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.964 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.329.984 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.081 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.097 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.112 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.128 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.147 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.238 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.254 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.272 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.286 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.307 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.396 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.412 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.428 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.443 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.461 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.553 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.569 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.586 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.601 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.620 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.707 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.724 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.745 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.760 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.781 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.876 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.893 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.909 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.925 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.330.947 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.036 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.052 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.068 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.083 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.102 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.190 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.206 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.223 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.238 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.258 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.347 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.363 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.379 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.396 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.416 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.510 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.529 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.546 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.571 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.592 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.681 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.698 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.717 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.734 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.754 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.844 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.862 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.877 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.893 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.331.914 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.008 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.023 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.040 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.055 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.074 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.173 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.196 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.213 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.229 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.249 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.338 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.355 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.370 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.385 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.414 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.506 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.525 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.539 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.555 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.577 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.674 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.690 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.705 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.719 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.737 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.829 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.845 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.862 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.876 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.895 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.332.984 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.002 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.017 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.031 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.051 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.141 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.158 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.174 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.189 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.208 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.299 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.323 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.340 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.355 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.375 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.465 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.482 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.497 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.511 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.529 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.617 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.635 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.650 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.664 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.683 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.773 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.792 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.808 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.822 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.841 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.930 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.947 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.963 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.976 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.333.996 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.088 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.105 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.127 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.141 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.160 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.250 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.268 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.283 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.300 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.321 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.412 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.429 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.444 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.458 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.477 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.570 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.587 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.604 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.618 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.637 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.727 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.743 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.758 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.773 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.796 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.886 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.903 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.918 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.942 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.334.964 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.054 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.071 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.085 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.101 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.120 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.213 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.230 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.245 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.260 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.282 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.374 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.391 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.410 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.426 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.445 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.536 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.552 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.567 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.582 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.602 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.692 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.709 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.722 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.737 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.769 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.860 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.877 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.892 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.909 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.335.930 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.020 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.037 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.053 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.068 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.088 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.192 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.212 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.231 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.246 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.267 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.354 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.372 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.387 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.401 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.420 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.509 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.526 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.542 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.557 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.577 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.672 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.698 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.715 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.730 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.749 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.841 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.859 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.874 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.888 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.909 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.336.998 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.016 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.031 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.046 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.066 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.155 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.172 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.187 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.202 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.222 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.311 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.328 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.343 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.358 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.378 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.467 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.486 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.501 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.524 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.544 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.631 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.648 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.663 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.682 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.703 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.796 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.813 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.829 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.845 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.866 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.959 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.977 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.337.992 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.010 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.033 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.126 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.145 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.160 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.177 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.200 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.291 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.308 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.323 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.338 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.365 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.456 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.473 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.488 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.506 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.528 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.622 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.640 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.656 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.672 [runtime.cc:1673] 619761 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.695 [elf.cc:566] 619761 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.785 [runtime.cc:1688] 619761 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.802 [runtime.cc:1953] 619761 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.816 [runtime.cc:1897] 619761 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619443,python3.9):2025-04-03-09:35:50.338.837 [devmm_svm.c:1972][ascend][curpid:619443,619761][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.338.854 [hdc.cc:83] [619761][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.870 [api_impl.cc:1990] 619761 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.338.896 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.339.520 [api_impl.cc:1990] 619761 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.339.540 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.339.561 [api_impl.cc:2664] 619761 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619443,python3.9):2025-04-03-09:35:50.340.253 [devmm_svm.c:1972][ascend][curpid:619443,619761][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.340.285 [hdc.cc:83] [619761][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.305 [api_impl.cc:1990] 619761 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.334 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.368 [api_impl.cc:2096] 619761 HostMalloc: size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.406 [api_error.cc:1260] 619761 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.687 [api_impl.cc:1990] 619761 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.772 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.795 [api_impl.cc:2664] 619761 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.939 [api_impl.cc:1990] 619761 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.961 [api_error.cc:1218] 619761 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.340.980 [api_impl.cc:2664] 619761 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.015 [peterson_lock.cc:69] [619761][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.032 [peterson_lock.cc:47] [619761][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.054 [hccl_comm.cc:129] [619761]hcclCommInitInfo:commId[hccl-619443-0-1743644148-hccl_world_group], rank[6], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[6], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.078 [hccl_comm.cc:175] [619761][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.095 [topo_matcher.cc:528] [619761][SetAivMode]AivMode is set to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.108 [topo_matcher.cc:540] [619761][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.126 [hcom_plugin.cc:349] [619761][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.341.163 [runtime.cc:5239] 619761 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.180 [adapter_rts.cc:198] [619761][hrtGetDeviceRefresh]deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.198 [hcom_common.cc:85] [619761][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.218 [op_base.cc:128] [619761][HcclGetDeviceId] deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.233 [hccl_comm.cc:1296] [619761]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffeb8f9ccb8] topoSize[2] [INFO] GE(619443,python3.9):2025-04-03-09:35:50.341.256 [hcom_topo_info.cc:24]619761 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.272 [hcom_plugin.cc:371] [619761][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.296 [topoinfo_detect.cc:555] [619761]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.312 [topoinfo_detect.cc:321] [619761]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.324 [topoinfo_detect.cc:219] [619761]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.344 [op_base.cc:1050] [619761][InitCommRootInfo]Init success, rankNum[8], rank[6], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.375 [profiler_base.cc:342] [619761]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.391 [op_base.cc:1107] [619761][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4981178]us, rankNum[8], rank[6] [WARNING] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.341.447 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.341.469 [op_base.cc:853] [619761]HcclGetCommName input handle=0xfffe90022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.341.498 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 6 [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.341.520 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 6, group name: hccl_world_group [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.341.604 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.341.624 [device.cpp:183]619761 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 6. [INFO] DEVICE(619443,fffe977eefa0,python3.9):2025-04-03-09:35:50.341.641 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 6, group name: hccl_world_group [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.341.646 [api_impl.cc:3419] 619761 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.341.668 [runtime.cc:3444] 619761 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.341.684 [api_impl.cc:3449] 619761 DeviceReset: Succ, userDeviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.341.694 [device.cpp:189]619761 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 6. [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.341.715 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:50.341.783 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5083.97 msec. [WARNING] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:50.341.818 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.341.888 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5585.29 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.341.936 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5585.35 msec. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.341.959 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.341.983 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9753.31 msec. [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.342.031 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.601 [device.cpp:143]619443 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.637 [api_impl.cc:2999] 619443 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.653 [device.cc:306] 619443 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.670 [runtime.cc:3393] 619443 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.686 [runtime.cc:3560] 619443 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.699 [api_impl.cc:2987] 619443 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.712 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.730 [api_impl.cc:2992] 619443 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.747 [api_impl.cc:3025] 619443 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.760 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.774 [acl.cpp:121]619443 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.787 [device.cpp:154]619443 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.356.823 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.842 [context.cpp:174]619443 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.356.865 [api_impl.cc:6913] 619443 CtxSetSysParamOpt: curCtx = 0xaaab04034d70, configOpt=0, configVal=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.356.879 [context.cpp:158]619443 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.357.431 [op_base.cc:1214] [619443][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.357.454 [op_base.cc:128] [619443][HcclGetDeviceId] deviceLogicId[6] [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.357.475 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.357.493 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.357.506 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.357.544 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.044 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.425 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.488 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.633 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 150.22 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.672 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.713 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 38.18 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.737 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.760 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 23.36 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.779 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.835 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 52.75 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.866 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.445 msec. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.936 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.358.989 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 5.64 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.048 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 33.89 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.090 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.152 msec. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.116 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.153 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.197 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 41.99 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.217 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.248 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 28.42 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.279 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.301 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 20.97 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.319 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.352 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 30.68 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.393 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.273 msec. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.426 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.518 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.584 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 64.08 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.638 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 27.93 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.678 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 18.41 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.728 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 25.63 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.787 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 33.69 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 96.74 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.936 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.72 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.359.954 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.360.018 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 44.07 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.360.071 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 29.53 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.360.093 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.297 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 45.94 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.802 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 450.141 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.826 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.882 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 35.74 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.935 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 2.503 msec. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.361.952 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.003 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 3.065 msec. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.366 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 47.34 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.393 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.483 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 87.55 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.505 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.557 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 49.44 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.577 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.608 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 28.28 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.643 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.662 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 34.29 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 7.28 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.736 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.759 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 50.31 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.777 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.797 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 19.45 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.826 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.845 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 19.5 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.893 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.915 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 49.69 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.934 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.954 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 19.17 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.972 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.362.989 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 17.08 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.005 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.048 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 36.7 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.098 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 50.08 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.139 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.155 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 15.35 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.171 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.189 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 16.84 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.205 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.221 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 15.24 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.239 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.255 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 14.87 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.298 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 32.94 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.334 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.400 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 63.25 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.438 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 37.11 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.474 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.493 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 18.01 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.509 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.524 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 14.21 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.543 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.560 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 16.27 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.578 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.598 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 20.05 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.629 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.649 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 31.58 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.678 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.695 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 28.28 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.712 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.729 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 16.28 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.745 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.769 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 23.17 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.363.854 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 64.87 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.034 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 153.09 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.058 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.112 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 32.62 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.247 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 89.43 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.274 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.325 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 29.46 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.348 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.392 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 25.07 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.409 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.449 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 21.86 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.467 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.539 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 52.47 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.561 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.621 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 39.56 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.642 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.766 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 89.29 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.789 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.863 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 52.4 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.886 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.364.906 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.019 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 112.25 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.042 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.064 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.181 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 115.421 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.203 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.292 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 69.15 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.315 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 51.81 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.415 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.432 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.449 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.467 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.484 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.512 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.528 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.544 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.559 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.575 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.594 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.613 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.628 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.645 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.659 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.916 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 233.221 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.941 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.962 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.365.980 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.001 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.020 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.112 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 71.19 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.161 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.181 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.256 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 53.35 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.313 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.265 msec. [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.600 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.366.734 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.008 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 211.76 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.128 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 31.35 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.232 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 73.9 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.271 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 12.34 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.323 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 26.51 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.407 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 57.58 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.461 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 28.52 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.508 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 23.07 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.590 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 56.73 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.633 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 19.46 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.683 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.611 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.734 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.001 msec. [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.367.780 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.092 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.136 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.375 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.195 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.220 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 41.08 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.411 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 56.82 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.478 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 41.01 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.570 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 65.271 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.611 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.669 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 35.87 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.705 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 11.54 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.727 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 0.7 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.773 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.529 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.823 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.581 msec. [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.864 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.972 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.368.996 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 0.75 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.038 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 19.27 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.074 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 4.27 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.097 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.38 us [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.130 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.208 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 88.031 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.233 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 0.82 us [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.258 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.312 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.308 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.458 msec. [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.342 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.422 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.087 msec. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.369.868 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.369.898 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.369.919 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.369.960 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.299 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.345 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.423 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.524 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.112 msec. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.606 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.653 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.678 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.697 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.729 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.748 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.370.795 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.371.002 [context.cpp:72]619795 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.371.038 [api_impl.cc:3717] 619795 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.371.059 [awatchdog_core.c:175](tid:619795) create watchdog for id : 65543, tid : 619795, timeout : 300s [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.371.075 [context.cpp:80]619795 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.371.106 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.371.122 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.371.161 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.371.185 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.372.597 [api_impl.cc:7115] 619795 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5a00000 [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.379.139 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.184 [api_impl.cc:7129] 619795 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.311 [api_impl.cc:7145] 619795 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.405 [device.cpp:143]619795 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.428 [api_impl.cc:2999] 619795 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.443 [device.cc:306] 619795 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.461 [runtime.cc:3393] 619795 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.479 [runtime.cc:3560] 619795 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.494 [api_impl.cc:2987] 619795 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.508 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.545 [api_impl.cc:2992] 619795 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.564 [api_impl.cc:3025] 619795 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.579 [acl.cpp:115]619795 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.594 [acl.cpp:121]619795 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.611 [device.cpp:154]619795 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.379.647 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.666 [context.cpp:174]619795 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.687 [api_impl.cc:6913] 619795 CtxSetSysParamOpt: curCtx = 0xaaab04034d70, configOpt=0, configVal=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.703 [context.cpp:158]619795 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.379.765 [op_base.cc:1214] [619795][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.788 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.379.802 [adapter_rts.cc:234] [619795][hrtGetDevice]deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:50.379.814 [op_base.cc:128] [619795][HcclGetDeviceId] deviceLogicId[6] [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.850 [context.cpp:72]619795 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.869 [api_impl.cc:3717] 619795 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.379.884 [context.cpp:80]619795 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.912 [api_error.cc:1807] 619795 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.934 [memory_task.cc:342] 619795 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.948 [memory_task.cc:473] 619795 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.960 [memory_task.cc:518] 619795 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.979 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.379.998 [memory_task.cc:1007] 619795 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaab107c88f0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.011 [memory_task.cc:1166] 619795 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.024 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.039 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.058 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.084 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.380.111 [callback.cpp:97]619795 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.144 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.184 [task_info.cc:521] 619795 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.202 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.218 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.233 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.246 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.380.259 [callback.cpp:109]619795 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.283 [npu_driver.cc:3844] 619594 CqReportGet: cq report get start, device_id=6, tsId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.307 [npu_driver.cc:3851] 619594 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.322 [api_impl.cc:5087] 619594 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.380.343 [callback.cpp:138]619594 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.380.355 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.380.358 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.380.381 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.380.387 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.380.411 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.381.755 [api_impl.cc:7115] 619795 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12d100000000, end ptr=0x12df35a00000 [INFO] PRE_ACT(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.388.749 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.388.783 [api_impl.cc:7129] 619795 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.388.883 [api_impl.cc:7145] 619795 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.389.019 [device.cpp:231]619795 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.040 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.389.058 [acl.cpp:802]619795 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.168 [npu_driver.cc:404] 619795 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.207 [stream.cc:2246] 619795 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.329 [stream_sqcq_manage.cc:147] 619795 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.348 [npu_driver.cc:882] 619795 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.875 [npu_driver.cc:6125] 619795 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.928 [npu_driver.cc:6137] 619795 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=310, streamId=6, logicCq=310, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.955 [npu_driver.cc:6189] 619795 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=6, logicCq=310. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.976 [npu_driver.cc:5916] 619795 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec025000, addr=dfffec025000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.389.989 [stream.cc:653] 619795 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.035 [stream.cc:276] 619795 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.052 [stream.cc:688] 619795 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=6, streamResId=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.390.071 [acl_resource_manager.cpp:814]619795 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.390.088 [acl_resource_manager.cpp:816]619795 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.239 [engine.cc:188] 619795 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.262 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.283 [task_info.cc:409] 619795 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.298 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.313 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.335 [npu_driver.cc:751] 619795 SqTaskSend: device_id=6, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.354 [stars_engine.cc:893] 619795 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.371 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.388 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.404 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.433 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.449 [logger.cc:152] 619795 TaskFinished: device_id=6, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.473 [stars_engine.cc:437] 619795 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.489 [stream.cc:2886] 619795 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.503 [stream.cc:852] 619795 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.519 [stream.cc:938] 619795 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.534 [stream.cc:1507] 619795 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.569 [stream_sqcq_manage.cc:451] 619795 FreeLogicCq: Return(0), threadIdentifier(281469490163616), devId(6), tsId(0), cqId(310), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.592 [npu_driver.cc:6163] 619795 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=6, logicCq=310. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.625 [npu_driver.cc:1180] 619795 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=310, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.643 [stream.cc:237] 619795 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.659 [stream_sqcq_manage.cc:216] 619795 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.390.672 [npu_driver.cc:966] 619795 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.391.394 [npu_driver.cc:974] 619795 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.391.414 [stream_sqcq_manage.cc:228] 619795 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.391.431 [npu_driver.cc:432] 619795 StreamIdFree: stream id free begin, device_id=6, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.391.450 [npu_driver.cc:441] 619795 StreamIdFree: Stream id free success, stream_id=6, device_id=6, tsId=0, streamFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.391.468 [device_msg_handler.cc:110] 619795 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.392.190 [device_msg_handler.cc:127] 619795 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.392.253 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.392.289 [npu_driver.cc:500] 619795 GetFaultEvent: get fault event, device_id=6, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.392.373 [acl.cpp:782]619795 GetFaultEventInfo: can not get fault event of device 6, runtime errorCode is 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:50.392.407 [error_manager.cc:989]619795 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.392.773 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.392.820 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.392.860 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.392.877 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.393.223 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.331 [context.cpp:72]619795 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.356 [api_impl.cc:3717] 619795 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.372 [context.cpp:80]619795 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.394 [api_error.cc:1807] 619795 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.412 [memory_task.cc:342] 619795 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.393.413 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.427 [memory_task.cc:473] 619795 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.445 [memory_task.cc:518] 619795 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.459 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.478 [memory_task.cc:1007] 619795 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaab107c9310, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.493 [memory_task.cc:1166] 619795 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.508 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.522 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.541 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.556 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.576 [callback.cpp:97]619795 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.597 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.614 [task_info.cc:521] 619795 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.629 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.654 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.670 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.688 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.703 [callback.cpp:109]619795 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.711 [npu_driver.cc:3844] 619594 CqReportGet: cq report get start, device_id=6, tsId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.728 [npu_driver.cc:3851] 619594 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.740 [api_impl.cc:5087] 619594 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.759 [callback.cpp:138]619594 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.774 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.783 [device.cpp:231]619795 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.788 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.800 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.393.816 [acl.cpp:802]619795 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.850 [npu_driver.cc:404] 619795 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.880 [stream.cc:2246] 619795 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.919 [stream_sqcq_manage.cc:147] 619795 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.393.936 [npu_driver.cc:882] 619795 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.190 [npu_driver.cc:6125] 619795 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.232 [npu_driver.cc:6137] 619795 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=311, streamId=7, logicCq=311, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.255 [npu_driver.cc:6189] 619795 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=7, logicCq=311. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.272 [npu_driver.cc:5916] 619795 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec025000, addr=dfffec025000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.285 [stream.cc:653] 619795 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.325 [stream.cc:276] 619795 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.342 [stream.cc:688] 619795 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=6, streamResId=65534 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.394.358 [acl_resource_manager.cpp:814]619795 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.394.371 [acl_resource_manager.cpp:816]619795 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.422 [engine.cc:188] 619795 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.442 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.457 [task_info.cc:409] 619795 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.469 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.483 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.500 [npu_driver.cc:751] 619795 SqTaskSend: device_id=6, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.513 [stars_engine.cc:893] 619795 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.528 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.544 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.558 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.572 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.585 [logger.cc:152] 619795 TaskFinished: device_id=6, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.602 [stars_engine.cc:437] 619795 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.615 [stream.cc:2886] 619795 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.629 [stream.cc:852] 619795 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.642 [stream.cc:938] 619795 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.655 [stream.cc:1507] 619795 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.677 [stream_sqcq_manage.cc:451] 619795 FreeLogicCq: Return(0), threadIdentifier(281469490163616), devId(6), tsId(0), cqId(311), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.696 [npu_driver.cc:6163] 619795 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=7, logicCq=311. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.725 [npu_driver.cc:1180] 619795 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=311, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.742 [stream.cc:237] 619795 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.755 [stream_sqcq_manage.cc:216] 619795 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.394.767 [npu_driver.cc:966] 619795 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.225 [npu_driver.cc:974] 619795 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.243 [stream_sqcq_manage.cc:228] 619795 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.266 [npu_driver.cc:432] 619795 StreamIdFree: stream id free begin, device_id=6, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.285 [npu_driver.cc:441] 619795 StreamIdFree: Stream id free success, stream_id=7, device_id=6, tsId=0, streamFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.301 [device_msg_handler.cc:110] 619795 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.623 [device_msg_handler.cc:127] 619795 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.658 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.395.679 [npu_driver.cc:500] 619795 GetFaultEvent: get fault event, device_id=6, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.395.739 [acl.cpp:782]619795 GetFaultEventInfo: can not get fault event of device 6, runtime errorCode is 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:50.395.758 [error_manager.cc:989]619795 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.396.058 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.396.094 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.396.112 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.396.128 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.396.213 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.301 [context.cpp:72]619795 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.324 [api_impl.cc:3717] 619795 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.341 [context.cpp:80]619795 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.360 [api_error.cc:1807] 619795 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.379 [memory_task.cc:342] 619795 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.391 [memory_task.cc:473] 619795 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.403 [memory_task.cc:518] 619795 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.416 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.431 [memory_task.cc:1007] 619795 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaab107c9d80, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.445 [memory_task.cc:1166] 619795 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.474 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.488 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.504 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.520 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.539 [callback.cpp:97]619795 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.556 [task_res.cc:380] 619795 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.572 [task_info.cc:521] 619795 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.584 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.596 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.612 [task_submit.cc:573] 619795 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.628 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.640 [callback.cpp:109]619795 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.649 [npu_driver.cc:3844] 619594 CqReportGet: cq report get start, device_id=6, tsId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.664 [npu_driver.cc:3851] 619594 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.675 [api_impl.cc:5087] 619594 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.689 [callback.cpp:138]619594 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.703 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.711 [device.cpp:231]619795 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.714 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.729 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.396.743 [acl.cpp:802]619795 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.775 [npu_driver.cc:404] 619795 StreamIdAlloc: stream id alloc begin, device_id=6, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.804 [stream.cc:2246] 619795 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.839 [stream_sqcq_manage.cc:147] 619795 Alloc: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.396.863 [npu_driver.cc:882] 619795 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.207 [npu_driver.cc:6125] 619795 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.249 [npu_driver.cc:6137] 619795 LogicCqAllocateV2: success: deviceId=6, tsId=0, cqId=312, streamId=8, logicCq=312, cqeDepth=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.272 [npu_driver.cc:6189] 619795 StreamBindLogicCq: success: deviceId=6, tsId=0, streamId=8, logicCq=312. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.291 [npu_driver.cc:5916] 619795 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=ec025000, addr=dfffec025000size=4096 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.305 [stream.cc:653] 619795 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.342 [stream.cc:276] 619795 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.359 [stream.cc:688] 619795 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=6, streamResId=65534 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.397.377 [acl_resource_manager.cpp:814]619795 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.397.390 [acl_resource_manager.cpp:816]619795 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.434 [engine.cc:188] 619795 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.453 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.470 [task_info.cc:409] 619795 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.483 [stream.cc:1400] 619795 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.496 [stream.cc:2595] 619795 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.516 [npu_driver.cc:751] 619795 SqTaskSend: device_id=6, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.528 [stars_engine.cc:893] 619795 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.547 [stream.cc:1494] 619795 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.562 [npu_driver.cc:5547] 619795 GetSqHead: deviceId=6, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.576 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.592 [task.cc:185] 619795 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.606 [logger.cc:152] 619795 TaskFinished: device_id=6, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.622 [stars_engine.cc:437] 619795 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.638 [stream.cc:2886] 619795 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.652 [stream.cc:852] 619795 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.675 [stream.cc:938] 619795 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.688 [stream.cc:1507] 619795 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.708 [stream_sqcq_manage.cc:451] 619795 FreeLogicCq: Return(0), threadIdentifier(281469490163616), devId(6), tsId(0), cqId(312), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.727 [npu_driver.cc:6163] 619795 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=8, logicCq=312. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.753 [npu_driver.cc:1180] 619795 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=312, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.770 [stream.cc:237] 619795 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.783 [stream_sqcq_manage.cc:216] 619795 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.397.798 [npu_driver.cc:966] 619795 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.245 [npu_driver.cc:974] 619795 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.263 [stream_sqcq_manage.cc:228] 619795 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.278 [npu_driver.cc:432] 619795 StreamIdFree: stream id free begin, device_id=6, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.293 [npu_driver.cc:441] 619795 StreamIdFree: Stream id free success, stream_id=8, device_id=6, tsId=0, streamFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.309 [device_msg_handler.cc:110] 619795 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.634 [device_msg_handler.cc:127] 619795 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.668 [runtime.cc:5239] 619795 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.398.688 [npu_driver.cc:500] 619795 GetFaultEvent: get fault event, device_id=6, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.398.745 [acl.cpp:782]619795 GetFaultEventInfo: can not get fault event of device 6, runtime errorCode is 0 [INFO] GE(619443,python3.9):2025-04-03-09:35:50.398.765 [error_manager.cc:989]619795 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.399.060 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.399.094 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.399.115 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.399.129 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619443,fffeb8f9efa0,python3.9):2025-04-03-09:35:50.399.201 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:50.450.927 [awatchdog_monitor.c:160](tid:619475) thread 619761 not exist [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.500.185 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.500.220 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.500.237 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.500.255 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.500.277 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.500.296 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.500.316 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.181 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.184 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.183 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.185 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.185 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.564.189 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.312 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.333 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.352 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.366 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.368 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.381 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.402 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.424 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.404 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.412 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.440 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.443 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.450 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.478 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.480 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.490 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.511 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.569 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.526 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.541 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.547 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.557 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.585 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.595 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.608 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.615 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.627 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.637 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.647 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.564.657 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.667 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.687 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.707 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.713 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.718 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.564.725 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.916.183 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:50.916.221 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.916.239 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:50.916.257 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.916.280 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.916.296 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.916.317 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.972.097 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.132 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.972.153 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.972.197 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.217 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.236 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.281 [stars_engine.cc:643] 619443 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.324 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.343 [task_manager.cc:485] 619443 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.358 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.372 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.387 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.409 [stars_engine.cc:643] 619443 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.424 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.441 [task_manager.cc:485] 619443 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.455 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.472 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.499 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.517 [stars_engine.cc:643] 619443 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.532 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.546 [task_manager.cc:485] 619443 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.562 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.579 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.592 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.619 [stars_engine.cc:622] 619443 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=306, retCode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.635 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.649 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.972.662 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.972.680 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.714 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.731 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.757 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.776 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.788 [stream.cc:2886] 619443 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.801 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.815 [stream.cc:2886] 619443 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.829 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.842 [stream.cc:2886] 619443 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.858 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.872 [stream.cc:2886] 619443 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.885 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.899 [stream.cc:2886] 619443 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.920 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.934 [stream.cc:2886] 619443 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.949 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.962 [stream.cc:2886] 619443 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.975 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.972.988 [stream.cc:2886] 619443 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.004 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.019 [stream.cc:2886] 619443 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.033 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.045 [stream.cc:2886] 619443 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.060 [event.cc:310] 619443 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.094 [task_res.cc:380] 619443 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=6, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.115 [event.cc:249] 619443 UpdateLatestRecord: device_id=6, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.134 [event_task.cc:259] 619443 ConstructSqeForEventRecordTask: event_record: device_id=6, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.149 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.162 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.191 [task_submit.cc:573] 619443 AllocTaskAndSendStars: device_id=6, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.208 [stars_engine.cc:1122] 619443 SyncTask: Task Wait: device_id=6, stream_id=3, task_id=0, logicCqId=307. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.227 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.243 [stars_engine.cc:1071] 619443 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.261 [stars_engine.cc:1017] 619443 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.279 [stars_engine.cc:1027] 619443 ProcReport: taskType=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.293 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.308 [event.cc:249] 619443 UpdateLatestRecord: device_id=6, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.326 [event_task.cc:211] 619443 DoCompleteSuccessForEventRecordTask: event_record: device_id=6, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea57101eaa [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.354 [event.cc:122] 619443 TryFreeEventIdAndCheckCanBeDelete: device_id=6, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.372 [task_res.cc:323] 619443 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=6, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.394 [npu_driver.cc:5618] 619443 GetSqEnable: deviceId=6, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.409 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.424 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.440 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.484 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.505 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.523 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.539 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.556 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.571 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.588 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.605 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.618 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.633 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.654 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.668 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.701 [stars_engine.cc:622] 619443 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=307, retCode=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.719 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:50.973.733 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.746 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:50.973.763 [memory.cpp:421]619443 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.46s ============================================================================================================== 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340901] [debug] [619761] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340909] [debug] [619761] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340916] [debug] [619761] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340922] [debug] [619761] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.359692] [debug] [619761] [lcal_comm.cpp:254] InitCommMem 6/8, localRank_ : 6, localRankSize_ : 8 success [2025-04-03 09:35:45.359809] [info] [619761] [lcal_comm.cpp:259] LcalCommInit 6/8 success and extraFlag:0 commArgs_.localRank : 6 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.388981] [error] [619795] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.393755] [error] [619795] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.396687] [error] [619795] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.449 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.482 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:50.982.554 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619443:281473203355680,MainProcess):2025-04-03-09:35:50.982.704 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.783 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.857 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.913 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:50.982.937 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.012.185 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.012.217 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.012.237 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.012.253 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.012.272 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.012.291 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.012.309 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.092.639 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.092.740 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619443,ffff35b76fa0,python3.9):2025-04-03-09:35:51.092.832 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.092.927 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.092.949 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619443,ffff36386fa0,python3.9):2025-04-03-09:35:51.093.003 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.082 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.102 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.163 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619443,ffff34b56fa0,python3.9):2025-04-03-09:35:51.093.242 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.410 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.431 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619443,ffff35366fa0,python3.9):2025-04-03-09:35:51.093.484 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.593 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.614 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.646 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.093.667 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.093.816 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.093.842 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.093.861 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.093.887 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.093.907 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.093.929 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.093.946 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.093.964 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.093.976 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.093.996 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.016 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.033 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.050 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.067 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.094.078 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.094.101 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.120 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.133 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.149 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.162 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.094.176 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.094.187 [stream.cpp:131]619443 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.205 [api_impl.cc:1394] 619443 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.216 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.233 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.094.272 [api_error.cc:895] 619443 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.094.289 [stream.cpp:146]619443 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.335 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.409 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.483 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.502 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.520 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.721 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.094.744 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.095.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.095.655 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.095.674 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.095.693 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.095.712 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.016 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.041 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.058 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.123 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.144 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.096.200 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_6 [INFO] GE_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.104.979 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.105.016 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.105.073 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.091 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.128 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.148 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.164 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.183 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.195 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.225 [device.cpp:143]619443 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.248 [api_impl.cc:2999] 619443 SetDevice: device_id=6, deviceMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.265 [device.cc:306] 619443 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.279 [runtime.cc:3393] 619443 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.299 [runtime.cc:3560] 619443 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.315 [api_impl.cc:2987] 619443 NewDevice: SetCurRef = 0xaaab04034d70,userDeviceId=6,deviceId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.331 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.353 [api_impl.cc:2992] 619443 NewDevice: New device success, user_device_id=6, die_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.105.372 [api_impl.cc:3025] 619443 SetDevice: SetDevice success, curCtx=0xaaab04034d70, user_device_id=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.389 [acl.cpp:115]619443 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.406 [acl.cpp:121]619443 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.105.424 [device.cpp:154]619443 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 6 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:51.105.451 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:51.221.316 [awatchdog_monitor.c:160](tid:619475) thread 619795 not exist [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.189 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.189 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.270 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.318 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.312 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.323 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.338 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.347 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.372 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.390 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.399 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.405 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.409 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.418 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.426 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.430 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.434 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.442 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.446 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.453 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.459 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.461 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.467 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.474 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.489 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.494 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.513 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.522 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.526 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.535 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.396.542 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.551 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.555 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.579 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.396.582 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.396.993 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.397.104 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.397.121 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.397.135 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.397.149 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.397.160 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.397.172 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.428.181 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.428.214 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.428.232 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.428.247 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.428.265 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.428.280 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.428.296 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.524.169 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.524.195 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.524.209 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.524.226 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.524.241 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.524.255 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.524.280 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.532.745 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:51.532.791 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:51.532.811 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:51.532.825 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.532.845 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.532.937 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.940.169 [trs_cb_event.c:119][ascend][curpid:619443,619594][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =576; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:51.940.202 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.940.221 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:51.940.238 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.940.252 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:51.940.268 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:51.940.285 [npu_driver.cc:3828] 619594 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=576. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.036.180 [trs_cb_event.c:119][ascend][curpid:619443,619595][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=6, tid =577; ret=16) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.036.214 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.036.237 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.036.255 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.036.272 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.036.287 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.036.308 [npu_driver.cc:3828] 619595 CqReportIrqWait: Irq wait start, deviceId=6, tsId=0, groupId=577. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.177 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.183 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.186 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.186 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.193 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.228.210 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.283 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.302 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.304 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.318 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.325 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.331 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.341 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.347 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.342 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.355 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.362 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.368 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.377 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.400 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.385 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.404 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.411 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.416 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.427 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.428 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.438 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.443 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.453 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.475 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.485 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.494 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.514 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.528 [tensor_data_transfer.cpp:1054]619688 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is _npu_log [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.539 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.544 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.552 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.228.562 [tensor_data_transfer.cpp:1056]619688 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.573 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.584 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.597 [api_impl.cc:6121] 619688 MemQueuePeek: Start to peek queue on devId 6, qid is 0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.228.623 [npu_driver.cc:4330] 619688 MemQueuePeek: peek mem queue, deviceId=6, qid=0, timeout=800. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.289 [op_base.cc:2136] [619443]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619443,fffe977eefa0,python3.9):2025-04-03-09:35:52.343.306 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.343.321 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.341 [adapter_rts.cc:198] [619443][hrtGetDeviceRefresh]deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.358 [op_base.cc:2151] [619443][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.377 [op_base.cc:128] [619443][HcclGetDeviceId] deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.409 [op_base.cc:2168] [619443]com is not global com [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.430 [hcom_plugin.cc:381] [619443][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.472 [hccl_comm.cc:1375] [619443]UnRegistTaskAbortHandler begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.490 [task_abort_handler.cc:140] [619443]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.343.505 [api_impl.cc:4192] 619443 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.343.519 [runtime.cc:4698] 619443 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.537 [hccl_communicator.cc:2112] [619443][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.343.553 [hccl_communicator.cc:2114] [619443][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.581 [hccl_communicator.cc:2082] [619443]start to stop the backGround Thread [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.343.596 [heartbeat.cc:1203] [619443]Heartbeat has been destroyed [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.612 [heartbeat.cc:451] [619443][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.343.626 [heartbeat.cc:452] [619443]Heartbeat has been destroyed [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.641 [hccl_impl.cc:69] [619443]start hccl resources destruction:deviceLogicId[6] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.343.661 [heartbeat.cc:1203] [619443]Heartbeat has been destroyed [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.673 [heartbeat.cc:451] [619443][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.343.685 [heartbeat.cc:452] [619443]Heartbeat has been destroyed [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.343.790 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.343.930 [task_exception_handler.cc:1232] [619443]deInit taskFailCallback [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.344.386 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.344.410 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.344.448 [api_error.cc:1278] 619443 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.344.466 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.344.493 [hccl_socket_manager.cc:81] [619443][DeInit][Server]ip[192.168.2.193] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.344.510 [hccl_socket_manager.cc:83] [619443][DeInit][Server]ip[192.168.2.193] port[16666] [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.344.524 [hccl_socket.cc:232] [619443][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.344.537 [network_manager.cc:69] [619443]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.344.566 [network_manager.cc:458] [619443]NetworkManager devicePhyId_[6] vnic stopped. refcount[0] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:52.344.605 [ra_host.c:911]tid:619443,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[6], local_ip[192.168.2.193] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:52.344.972 [ra_host.c:404]tid:619443,ra_socket_deinit(404) : Input parameters: phy_id[6] family[2] local_ip[192.168.2.193] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.345.131 [hccl_socket.cc:68] [619443][HcclSocket][DeInit] device[6] stops listen on ip[192.168.2.193], port[16666] success, socketType[0]. [WARNING] HCCL(619443,python3.9):2025-04-03-09:35:52.345.153 [hccl_socket.cc:232] [619443][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.345.171 [hccl_communicator.cc:1741] [619443][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.345.188 [network_manager.cc:69] [619443]NetworkManager::GetInstance deviceLogicID[6]. [INFO] HCCP(619443,python3.9):2025-04-03-09:35:52.345.201 [ra_host.c:248]tid:619443,ra_deinit(248) : Input parameters: phy_id[6], nic_position:[1] [INFO] HCCP(619443,python3.9):2025-04-03-09:35:52.345.217 [ra_hdc.c:651]tid:619443,ra_hdc_deinit(651) : hdc deinit start! phy_id[6] [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.345.354 [hdc_client.c:417][ascend][curpid:619443,619443][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4271) [INFO] ROCE(619443,python3.9):2025-04-03-09:35:52.345.426 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619443,python3.9):2025-04-03-09:35:52.345.443 [ra_hdc.c:685]tid:619443,ra_hdc_deinit(685) : hdc deinit OK! phy_id[6] [INFO] ROCE(619443,python3.9):2025-04-03-09:35:52.345.455 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.506 [tsd_client.cpp:189][TsdCapabilityGet][tid:619443] TsdCapabilityGet Begin. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.522 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.535 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.549 [process_mode_manager.cpp:1042][CapabilityGet][tid:619443] [ProcessModeManager] enter into CapabilityGet process deviceId[6] type[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.345.562 [adapter_tdt.cc:66] [619443]Get TsdCapability success. deviceLogicId[6] [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.576 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.591 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.602 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619443] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.619 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619443] add close subproc:30824, proctype:0 [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.345.652 [hdc_client.cpp:154][TsdRecvData][tid:619443] recv timeout is:150000 ms [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.416.999 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619443] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.417.022 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619443] [TsdClient] DeviceMsgProc recvMsg realDeviceId[6] msgType[61] localDevId[6] rspCode[0]heterogeneousSubPid[30824], tsdSupportLevel_[63] [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.417.042 [hdc_client.cpp:172][TsdRecvData][tid:619443] [deviceId=6] the Tsd recv data already [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.417.057 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619443] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.417.071 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619443] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.082 [adapter_tdt.cc:52] [619443]Open TsdProcess success. deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.093 [network_manager.cc:405] [619443][DeInit]finish CloseHccpProcess, phy_id[6], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.106 [network_manager.cc:409] [619443]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.120 [hccl_communicator_attrs.cc:1307] [619443][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.131 [hccl_communicator_attrs.cc:1275] [619443]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.147 [notify_pool_impl.cc:43] [619443]NotifyPoolImpl Destroy. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.163 [notify_pool_impl.cc:56] [619443]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.183 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.204 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.216 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.233 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.245 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.256 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.269 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.282 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.300 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.313 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.329 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=0. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.349 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=0.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.366 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.376 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.387 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.398 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.410 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.422 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.437 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.449 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=1. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.465 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=1.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.480 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.492 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.503 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.513 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.523 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.537 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.551 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.564 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=2. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.579 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=2.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.606 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.619 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.630 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.641 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.654 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.667 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.681 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.693 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=3. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.711 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=3.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.729 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.740 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.752 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.767 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.778 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.792 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.806 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.819 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=4. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.836 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=4.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.852 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.865 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.878 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.890 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.903 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.919 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.417.933 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.948 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=5. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.417.968 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=5.) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.986 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.417.998 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.017 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.030 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.043 [api_impl.cc:3100] 619443 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.056 [api_impl.cc:3111] 619443 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.069 [api_impl.cc:3130] 619443 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.081 [p2p_mgmt.cc:171] [619443]local logic id:6, local physic id:6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.094 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.108 [api_impl.cc:3520] 619443 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.121 [p2p_mgmt.cc:178] [619443]disable p2p: local logic id:6, remote physic id:7. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.134 [api_impl.cc:3184] 619443 DisableP2P: Disable P2P devIdDes=6, phyIdSrc=7. [INFO] DRV(619443,python3.9):2025-04-03-09:35:52.418.150 [devdrv_pcie.c:890][ascend][curpid:619443,619443][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=6; peer_phy_devid=7.) [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.168 [hccl_communicator.cc:2043] [619443]Profiling is deinitiated. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.182 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.203 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.418.223 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.243 [opexecounter.cc:103] [619443]free counter mem resource [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.418.258 [atrace_api.c:118](tid:619443) AtraceDestroy start. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.418.274 [tracer_mgr_operate.c:248](tid:619443) destroy object HCCL_619761_6, exitSave(true). [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.418.290 [atrace_api.c:120](tid:619443) AtraceDestroy end. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.418.303 [adapter_trace.cc:33] [619443]Call TraceDestroy, Params:handle[0xaaaaf40448f8] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.148 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.207 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.233 [queue_notify_manager.cc:100] [619443]QueueNotifyManager Destroy. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.251 [queue_notify_manager.cc:106] [619443]QueueNotifyManager Destroy success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.382 [queue_notify_manager.cc:100] [619443]QueueNotifyManager Destroy. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.400 [queue_notify_manager.cc:106] [619443]QueueNotifyManager Destroy success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.466 [ccl_buffer_manager.cc:125] [619443][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.486 [ccl_buffer_manager.cc:147] [619443][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.501 [ccl_buffer_manager.cc:160] [619443][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.515 [ccl_buffer_manager.cc:163] [619443][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.538 [ccl_buffer_manager.cc:166] [619443][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.581 [op_base.cc:128] [619443][HcclGetDeviceId] deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.598 [op_base.cc:2196] [619443]op_base comm destroy complete, take time [77279]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[6]. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.420.614 [device.cpp:183]619443 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.631 [api_impl.cc:3419] 619443 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.646 [runtime.cc:3444] 619443 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.660 [api_impl.cc:3449] 619443 DeviceReset: Succ, userDeviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.420.675 [device.cpp:189]619443 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 6. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.420.720 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.420.747 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.420.762 [hcom_executor.cc:40] [619443]Hcom Excutor Finalize start. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.882 [api_impl.cc:3419] 619709 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.904 [runtime.cc:3444] 619709 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.907 [api_impl.cc:3419] 619707 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.919 [api_impl.cc:3449] 619709 DeviceReset: Succ, userDeviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.925 [runtime.cc:3444] 619707 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.420.937 [api_impl.cc:3449] 619707 DeviceReset: Succ, userDeviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.175 [api_impl.cc:3419] 619708 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.175 [api_impl.cc:3419] 619710 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.205 [runtime.cc:3444] 619708 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.216 [runtime.cc:3444] 619710 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.222 [api_impl.cc:3449] 619708 DeviceReset: Succ, userDeviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.231 [api_impl.cc:3449] 619710 DeviceReset: Succ, userDeviceId=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.351 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.421.373 [adapter_rts.cc:321] [619443][hrtCtxSetCurrent] success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.421.387 [stream.cc:182] [619443]Switch Ctx ret[0], curCtx[0xaaab04034d70], setCtx[0xaaab04034d70], stream id[5] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.407 [api_impl.cc:1228] 619443 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.421.426 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.421.476 [single_op_manager.cc:50]619443 ReleaseResource:ReleaseResource in. resource id = 0xaaab100459e0 [INFO] GE(619443,python3.9):2025-04-03-09:35:52.421.498 [caching_allocator.cc:402]619443 TryFreeBlocks:Try free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.421.515 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:52.421.544 [npu_memory_allocator.cc:72]619443 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.421.560 [stream_executor.cpp:197]619443 RemoveExecutor: To remove executor by stream = 187651684850144 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.421.588 [acl_resource_manager.cpp:823]619443 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.606 [stream.cc:852] 619443 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.624 [stream.cc:938] 619443 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.637 [stream.cc:1507] 619443 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.666 [stream_sqcq_manage.cc:451] 619443 FreeLogicCq: Return(0), threadIdentifier(281473203355680), devId(6), tsId(0), cqId(309), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.691 [npu_driver.cc:6163] 619443 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=5, logicCq=309. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.421.727 [npu_driver.cc:1180] 619443 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=309, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.424.415 [stream.cc:237] 619443 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.424.443 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.424.458 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.039 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.060 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.076 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.094 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=5, device_id=6, tsId=0, streamFlag=1536 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.425.771 [stream.cc:185] [619443][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab100459e0], stream id[5] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.791 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.425.806 [adapter_rts.cc:321] [619443][hrtCtxSetCurrent] success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.425.818 [stream.cc:193] [619443]Restore Ctx ret[0], setCtx[0xaaab04034d70], stream id[5] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.865 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.425.883 [adapter_rts.cc:321] [619443][hrtCtxSetCurrent] success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.425.896 [stream.cc:182] [619443]Switch Ctx ret[0], curCtx[0xaaab04034d70], setCtx[0xaaab04034d70], stream id[4] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.425.910 [api_impl.cc:1228] 619443 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.425.937 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.425.952 [single_op_manager.cc:50]619443 ReleaseResource:ReleaseResource in. resource id = 0xaaab0ffc9980 [INFO] GE(619443,python3.9):2025-04-03-09:35:52.425.967 [caching_allocator.cc:402]619443 TryFreeBlocks:Try free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.425.979 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:52.425.991 [npu_memory_allocator.cc:72]619443 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.426.003 [stream_executor.cpp:197]619443 RemoveExecutor: To remove executor by stream = 187651684342144 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.426.016 [acl_resource_manager.cpp:823]619443 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.028 [stream.cc:852] 619443 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.043 [stream.cc:938] 619443 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.056 [stream.cc:1507] 619443 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.078 [stream_sqcq_manage.cc:451] 619443 FreeLogicCq: Return(0), threadIdentifier(281473203355680), devId(6), tsId(0), cqId(308), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.099 [npu_driver.cc:6163] 619443 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=4, logicCq=308. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.426.131 [npu_driver.cc:1180] 619443 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=308, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.430 [stream.cc:237] 619443 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.451 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.466 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.926 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.947 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.963 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.428.984 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=4, device_id=6, tsId=0, streamFlag=1536 [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.577 [stream.cc:185] [619443][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab0ffc9980], stream id[4] [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.429.597 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.613 [adapter_rts.cc:321] [619443][hrtCtxSetCurrent] success [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.628 [stream.cc:193] [619443]Restore Ctx ret[0], setCtx[0xaaab04034d70], stream id[4] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.661 [hcom_executor.cc:45] [619443][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.429.702 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.429.741 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.770 [hcom_ops_kernel_info_store.cc:3423] [619443]finalize hccl kernel info store. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.802 [hcom_common.cc:85] [619443][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[6] [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.824 [plugin_manager.cc:56] [619443]hccl ops plugin finalize start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.842 [hcom_plugin.cc:90] [619443]Finalize start. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.934 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.951 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.967 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.429.982 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.007 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.023 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.035 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.047 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.071 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.088 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.101 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.113 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.139 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.156 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.168 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.181 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.206 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.222 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.234 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.246 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.258 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.273 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.285 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.301 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.314 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.330 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.342 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.356 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.391 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.406 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.419 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.436 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.461 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.474 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.486 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.498 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.519 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.531 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.543 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.554 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.575 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.590 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.601 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.613 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.634 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.647 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.659 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.671 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.693 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.707 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.718 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.729 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.754 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.770 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.783 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.795 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.825 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.841 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.854 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.866 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.891 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.910 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.921 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.934 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.946 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.958 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.969 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.979 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.430.991 [hcom_plugin.cc:119] [619443]Finalize: HcomDestroy success. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.003 [plugin_manager.cc:62] [619443]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.431.069 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.431.096 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.111 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.127 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.138 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.152 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.163 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.176 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.187 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.198 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.211 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.223 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.233 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.245 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.256 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.268 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.280 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.291 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.303 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.313 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.325 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.337 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.349 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.362 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.383 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.394 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.407 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.421 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.432 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.444 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.457 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.470 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.481 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.493 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.503 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.514 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.526 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.538 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.550 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.563 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.573 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.585 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.598 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.608 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.621 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.632 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.645 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.657 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.669 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.681 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.694 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.708 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.719 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.730 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.743 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.756 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.766 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.788 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.800 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.812 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.822 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.833 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.846 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.858 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.869 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.881 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.892 [hcom_common.cc:958] [619443][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.906 [hcom_common.cc:961] [619443]Entry-HcomDestroy:void skip [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.918 [hcom.cc:337] [619443][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:52.431.929 [hcom.cc:340] [619443]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.431.954 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.431.976 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.207 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.237 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.256 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.272 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.286 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.302 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.316 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.332 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.346 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.361 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.387 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.402 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.416 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.430 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.444 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.458 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.472 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.498 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.519 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.539 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.558 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.574 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.592 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.607 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.621 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.637 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.663 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.679 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.699 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.718 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.733 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.747 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.766 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe90100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.783 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe90101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.432.808 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.831 [callback.cpp:37]619443 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.853 [callback.cpp:44]619443 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.892 [context.cpp:72]619443 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.432.909 [api_impl.cc:3717] 619443 ContextSetCurrent: set context=aaab04034d70, flag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.923 [context.cpp:80]619443 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.938 [stream.cpp:75]619443 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619443,python3.9):2025-04-03-09:35:52.432.963 [stream_manage_func_registry.cc:22]619443 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.432.977 [stream.cpp:80]619443 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.432.991 [api_impl.cc:1228] 619443 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.004 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.433.019 [single_op_manager.cc:50]619443 ReleaseResource:ReleaseResource in. resource id = 0xaaab05c0c0c0 [INFO] GE(619443,python3.9):2025-04-03-09:35:52.433.042 [caching_allocator.cc:402]619443 TryFreeBlocks:Try free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.433.056 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:52.433.069 [npu_memory_allocator.cc:72]619443 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.080 [stream_executor.cpp:197]619443 RemoveExecutor: To remove executor by stream = 187651512647872 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.092 [acl_resource_manager.cpp:823]619443 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.106 [stream.cc:852] 619443 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.120 [stream.cc:938] 619443 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.137 [event.cc:1036] 619443 WaitForBusy: device_id=6, event_id=0 finished map already empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.156 [npu_driver.cc:626] 619443 EventIdFree: event id free begin, device_id=6, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.184 [raw_device.cc:1158] 619443 FreeEventIdFromDrv: device_id=6, event_id=0 tear down. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.200 [event.cc:61] 619443 ~Event: device_id=6, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.220 [npu_driver.cc:3803] 619443 SqCqFree: deviceId=6, tsId=0, sqId=91, cqId=91 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.257 [subscribe.cc:244] 619443 Delete: delete callback subscribe: groupId=576, threadId=281471327465376, devId=6, tsId=0, streamId=2, sqId=91, cqId=91 [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.433.258 [trs_sqcq.c:1491][ascend][curpid:619443,619594][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.275 [stream.cc:1507] 619443 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.283 [npu_driver.cc:3833] 619594 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.296 [api_impl.cc:5058] 619594 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.299 [stream_sqcq_manage.cc:451] 619443 FreeLogicCq: Return(0), threadIdentifier(281473203355680), devId(6), tsId(0), cqId(306), isFastCq(0). [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.311 [callback.cpp:132]619594 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.317 [npu_driver.cc:6163] 619443 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=2, logicCq=306. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.326 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.337 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.350 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.352 [npu_driver.cc:1180] 619443 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=306, drvFlag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.368 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.381 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.403 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.417 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.430 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.442 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.453 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.465 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.477 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.488 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.499 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.511 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.521 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.534 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.547 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.557 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.568 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.579 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.590 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.603 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.616 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.626 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.639 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.651 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.661 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.675 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.686 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.706 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.719 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.733 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.745 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.755 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.768 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.781 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.796 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.809 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.822 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.835 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.846 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.858 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.869 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.881 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.894 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.904 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.915 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.927 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.939 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.949 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.962 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.433.975 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.433.987 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.000 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.023 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.034 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.046 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.057 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.068 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.080 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.093 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.106 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.119 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.130 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.144 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.157 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.168 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.179 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.190 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.202 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.216 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.228 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.239 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.250 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.263 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.273 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.284 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.295 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.326 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.341 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.353 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.366 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.377 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.388 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.401 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.412 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.423 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.435 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.447 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.462 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.473 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.485 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.500 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.512 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.524 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.535 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.547 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.558 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.571 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.583 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.596 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.609 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.622 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.642 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.656 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.669 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.682 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.694 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.707 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.720 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.731 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.741 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.752 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.764 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.778 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.788 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.801 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.815 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.827 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.837 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.850 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.863 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.874 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.885 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.899 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.912 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.924 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.936 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.956 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.434.968 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.979 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.434.990 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.002 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.013 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.026 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.037 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.048 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.062 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.073 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.084 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.096 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.109 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.122 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.135 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.145 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.157 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.168 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.184 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.197 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.209 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.220 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.231 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.245 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.268 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.280 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.291 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.303 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.315 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.329 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.341 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.354 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.364 [stream.cc:237] 619443 FreeStreamId: Free stream_id=2. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.367 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.380 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.382 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.393 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=6, tsId=0, drvFlag=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.397 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.411 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.424 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.438 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.449 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.463 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.477 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.490 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.504 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.518 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.532 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.544 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.566 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.579 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.591 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.604 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.615 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.628 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.639 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.651 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.664 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.675 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.686 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.697 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.709 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.720 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.732 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.745 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.755 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.766 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.778 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.794 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.805 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.817 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.828 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.838 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.848 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.856 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=6, tsId=0, drvFlag=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.868 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.876 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.882 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.892 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=2, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.896 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.907 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=2, device_id=6, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.909 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.927 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.940 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.953 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.435.966 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.977 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.435.988 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.002 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.017 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.031 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.042 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.054 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.065 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.077 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.089 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.102 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.113 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.133 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.148 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.170 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.186 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.200 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.214 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.227 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.241 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.254 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.265 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.278 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.290 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.303 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.316 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.328 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.342 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.352 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.363 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.378 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.391 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.405 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.419 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.433 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.446 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.458 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.478 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.490 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.503 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.517 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.528 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.539 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.549 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.564 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.576 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.588 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.600 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.611 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.622 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.635 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.649 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.663 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.673 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.685 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.697 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.709 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.721 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.732 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.743 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.754 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.765 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.789 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.801 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.813 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.826 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.837 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.850 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.865 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.879 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.892 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.904 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.917 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.929 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.940 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.952 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.965 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.436.976 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.436.990 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.002 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.014 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.024 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.035 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.048 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.061 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.073 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.093 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.106 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.120 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.134 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.149 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.162 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.175 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.188 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.198 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.210 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.224 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.235 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.245 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.256 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.269 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.281 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.295 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.307 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.318 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.331 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.342 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.353 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.364 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.376 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.387 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.405 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.417 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.428 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.442 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.452 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.464 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.475 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.487 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.500 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.514 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.527 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.539 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.551 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.563 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.576 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.588 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.599 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.614 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.625 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.636 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.649 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.661 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.674 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.685 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.698 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.724 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.737 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.749 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.761 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.773 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.785 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.798 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.809 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.821 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.832 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.844 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.858 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.870 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.884 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.897 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.909 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.922 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.933 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.946 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.958 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.970 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.437.982 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.437.997 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.013 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.026 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.045 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.057 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.068 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.082 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.093 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.106 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.118 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.131 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.143 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.156 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.168 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.179 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.191 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.201 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.215 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.230 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.241 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.252 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.264 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.275 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.288 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.301 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.313 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.326 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.345 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.356 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.367 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.380 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.379 [stream.cpp:87]619443 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.393 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.406 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.421 [callback.cpp:116]619594 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.438.428 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab05c0c0c0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.435 [subscribe.cc:308] 619594 GetGroupIdByThreadId: can not find subscribe info, threadId=281471327465376 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.448 [api_impl.cc:5043] 619594 ProcessReport: get groupId fail, threadIdentifier=281471327465376, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.449 [stream.cpp:75]619443 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.459 [callback.cpp:130]619594 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] GE(619443,python3.9):2025-04-03-09:35:52.438.467 [stream_manage_func_registry.cc:22]619443 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.480 [stream.cpp:80]619443 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.495 [api_impl.cc:1228] 619443 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.509 [acl_resource_manager.cpp:814]619443 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] DEVICE(619443,ffff267cefa0,python3.9):2025-04-03-09:35:52.438.513 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.438.525 [single_op_manager.cc:50]619443 ReleaseResource:ReleaseResource in. resource id = 0xaaab05d3c720 [INFO] GE(619443,python3.9):2025-04-03-09:35:52.438.541 [caching_allocator.cc:402]619443 TryFreeBlocks:Try free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:52.438.552 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:52.438.563 [npu_memory_allocator.cc:72]619443 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.575 [stream_executor.cpp:197]619443 RemoveExecutor: To remove executor by stream = 187651513894688 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.586 [acl_resource_manager.cpp:823]619443 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.599 [stream.cc:852] 619443 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.613 [stream.cc:938] 619443 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.637 [event.cc:1036] 619443 WaitForBusy: device_id=6, event_id=1 finished map already empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.653 [npu_driver.cc:626] 619443 EventIdFree: event id free begin, device_id=6, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.676 [raw_device.cc:1158] 619443 FreeEventIdFromDrv: device_id=6, event_id=1 tear down. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.689 [event.cc:61] 619443 ~Event: device_id=6, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.705 [npu_driver.cc:3803] 619443 SqCqFree: deviceId=6, tsId=0, sqId=92, cqId=92 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.737 [subscribe.cc:244] 619443 Delete: delete callback subscribe: groupId=577, threadId=281471319011232, devId=6, tsId=0, streamId=3, sqId=92, cqId=92 [WARNING] DRV(619443,python3.9):2025-04-03-09:35:52.438.742 [trs_sqcq.c:1491][ascend][curpid:619443,619595][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=6; type=1; ret=16) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.754 [stream.cc:1507] 619443 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.768 [npu_driver.cc:3833] 619595 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=6, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.784 [api_impl.cc:5058] 619595 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.784 [stream_sqcq_manage.cc:451] 619443 FreeLogicCq: Return(0), threadIdentifier(281473203355680), devId(6), tsId(0), cqId(307), isFastCq(0). [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.803 [callback.cpp:132]619595 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.811 [npu_driver.cc:6163] 619443 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=3, logicCq=307. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.817 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.833 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.845 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.845 [npu_driver.cc:1180] 619443 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=307, drvFlag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.857 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.870 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.886 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.898 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.911 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.923 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.934 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.946 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.974 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.438.986 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.438.998 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.012 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.024 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.039 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.052 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.064 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.077 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.087 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.098 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.111 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.122 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.134 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.146 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.156 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.167 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.181 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.193 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.206 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.216 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.227 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.239 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.251 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.262 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.285 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.297 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.308 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.322 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.335 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.345 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.357 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.367 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.379 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.392 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.407 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.419 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.430 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.441 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.452 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.466 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.479 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.493 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.505 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.516 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.526 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.537 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.552 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.564 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.576 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.595 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.605 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.616 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.630 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.643 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.656 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.668 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.678 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.693 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.705 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.716 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.728 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.739 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.752 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.764 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.776 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.790 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.803 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.816 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.827 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.839 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.851 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.863 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.876 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.895 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.907 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.920 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.931 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.944 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.954 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.964 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.439.980 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.439.993 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.007 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.018 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.029 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.041 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.054 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.066 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.076 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.089 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.100 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.112 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.123 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.136 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.147 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.168 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.186 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.198 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.216 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.230 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.242 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.253 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.264 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.275 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.288 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.302 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.312 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.326 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.338 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.349 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.361 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.375 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.388 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.398 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.409 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.420 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.432 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.447 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.462 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.476 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.487 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.500 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.511 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.532 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.546 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.558 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.568 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.578 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.590 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.603 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.618 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.630 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.639 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.651 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.664 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.677 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.689 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.702 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.713 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.725 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.735 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.746 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.759 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.772 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.785 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.798 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.811 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.825 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.848 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.862 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.873 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.885 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.897 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.910 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.923 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.935 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.948 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.959 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.440.971 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.982 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.440.998 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.012 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.022 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.032 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.044 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.056 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.069 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.083 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.094 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.106 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.118 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.130 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.151 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.164 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.174 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.189 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.203 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.215 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.226 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.240 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.251 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.258 [stream.cc:237] 619443 FreeStreamId: Free stream_id=3. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.262 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.272 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.277 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.284 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=6, tsId=0, drvFlag=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.289 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.303 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.317 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.331 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.343 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.354 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.365 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.377 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.390 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.402 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.414 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.424 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.456 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.470 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.484 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.496 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.507 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.516 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.526 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.539 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.551 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.563 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.576 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.587 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.598 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.611 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.622 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.633 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.643 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.654 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.667 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.677 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.688 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.700 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.715 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.727 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.731 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=6, tsId=0, drvFlag=0 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.745 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.753 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.758 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.768 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.776 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.785 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=3, device_id=6, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.790 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.804 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.814 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.828 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.838 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.850 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.863 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.873 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.884 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.896 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.906 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.918 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.929 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.942 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.953 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.964 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.441.975 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.441.988 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.005 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.019 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.030 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.042 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.051 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.064 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.078 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.090 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.102 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.113 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.126 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.138 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.148 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.160 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.170 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.181 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.192 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.204 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.214 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.225 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.237 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.248 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.262 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.277 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.298 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.309 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.320 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.333 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.344 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.356 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.367 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.377 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.387 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.398 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.408 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.422 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.432 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.443 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.454 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.465 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.478 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.491 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.501 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.512 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.523 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.536 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.549 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.561 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.573 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.596 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.610 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.622 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.633 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.645 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.657 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.670 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.681 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.693 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.705 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.717 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.731 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.744 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.758 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.771 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.783 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.795 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.805 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.816 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.830 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.842 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.853 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.864 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.875 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.887 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.905 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.916 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.928 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.940 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.953 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.965 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.442.975 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.986 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.442.998 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.009 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.023 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.036 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.047 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.059 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.071 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.086 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.099 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.111 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.122 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.132 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.143 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.154 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.166 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.178 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.188 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.208 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.219 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.234 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.248 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.258 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.269 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.283 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.295 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.305 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.317 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.329 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.340 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.353 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.366 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.380 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.393 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.405 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.418 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.433 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.445 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.458 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.471 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.484 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.496 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.507 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.525 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.537 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.548 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.561 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.572 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.584 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.595 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.607 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.619 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.633 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.643 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.654 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.665 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.676 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.679 [event.cc:61] 619443 ~Event: device_id=6, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.687 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.706 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.717 [stream.cpp:87]619443 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.720 [callback.cpp:116]619595 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.736 [subscribe.cc:308] 619595 GetGroupIdByThreadId: can not find subscribe info, threadId=281471319011232 [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.750 [api_impl.cc:5043] 619595 ProcessReport: get groupId fail, threadIdentifier=281471319011232, reCode=0x7160004 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.443.754 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab05d3c720. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.763 [callback.cpp:130]619595 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.774 [device.cpp:183]619443 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.803 [api_impl.cc:3419] 619443 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.817 [runtime.cc:3444] 619443 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] DEVICE(619443,ffff25fbefa0,python3.9):2025-04-03-09:35:52.443.823 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.443.831 [api_impl.cc:3449] 619443 DeviceReset: Succ, userDeviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.847 [device.cpp:189]619443 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 6. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.443.873 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.443.913 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.443.940 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.443.957 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.969 [callback.cpp:37]619443 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:52.443.985 [callback.cpp:44]619443 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.444.010 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:52.444.035 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.565.987 [awatchdog_monitor.c:160](tid:619475) thread 619707 not exist [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.566.016 [awatchdog_monitor.c:160](tid:619475) thread 619708 not exist [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.566.032 [awatchdog_monitor.c:160](tid:619475) thread 619709 not exist [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:52.566.049 [awatchdog_monitor.c:160](tid:619475) thread 619710 not exist [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.748.717 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.748.750 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.748.767 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:52.748.781 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.748.802 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:52.748.894 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.056.186 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.056.188 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.056.191 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.305 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.327 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.347 [tensor_data_transfer.cpp:1054]619694 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.351 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.363 [tensor_data_transfer.cpp:1056]619694 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.371 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.376 [api_impl.cc:6121] 619694 MemQueuePeek: Start to peek queue on devId 6, qid is 2. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.385 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.385 [tensor_data_transfer.cpp:1054]619698 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.390 [npu_driver.cc:4330] 619694 MemQueuePeek: peek mem queue, deviceId=6, qid=2, timeout=800. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.400 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.407 [tensor_data_transfer.cpp:1056]619698 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.420 [tensor_data_transfer.cpp:1054]619700 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.424 [api_impl.cc:6121] 619698 MemQueuePeek: Start to peek queue on devId 6, qid is 4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.056.432 [tensor_data_transfer.cpp:1056]619700 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.437 [npu_driver.cc:4330] 619698 MemQueuePeek: peek mem queue, deviceId=6, qid=4, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.446 [api_impl.cc:6121] 619700 MemQueuePeek: Start to peek queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.056.459 [npu_driver.cc:4330] 619700 MemQueuePeek: peek mem queue, deviceId=6, qid=5, timeout=800. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.060.173 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.060.174 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.060.175 [queue_client_comm.c:114][ascend][curpid:619443,619688][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.265 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.284 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.292 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.303 [tensor_data_transfer.cpp:1054]619696 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_image_summary [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.309 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.316 [tensor_data_transfer.cpp:1056]619696 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.320 [npu_driver.cc:4338] 619688 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.323 [tensor_data_transfer.cpp:1054]619692 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.327 [api_impl.cc:6121] 619696 MemQueuePeek: Start to peek queue on devId 6, qid is 3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.336 [tensor_data_transfer.cpp:612]619688 acltdtReceiveTensorV2: queue is empty, device is 6, name is _npu_log [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.340 [tensor_data_transfer.cpp:1056]619692 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.346 [npu_driver.cc:4330] 619696 MemQueuePeek: peek mem queue, deviceId=6, qid=3, timeout=800. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.358 [api_impl.cc:6121] 619692 MemQueuePeek: Start to peek queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.370 [npu_driver.cc:4330] 619692 MemQueuePeek: peek mem queue, deviceId=6, qid=1, timeout=800. [INFO] DEVICE(619443,fffef447dfa0,python3.9):2025-04-03-09:35:53.060.442 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.543 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is _npu_log [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.565 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.060.578 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=0. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.060.988 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is _npu_log [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.061.026 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.061.050 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.888.172 [queue_client_comm.c:114][ascend][curpid:619443,619692][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.888.196 [queue_client_comm.c:114][ascend][curpid:619443,619694][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.888.206 [queue_client_comm.c:114][ascend][curpid:619443,619700][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.888.210 [queue_client_comm.c:114][ascend][curpid:619443,619698][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.254 [npu_driver.cc:4338] 619692 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.888.340 [tensor_data_transfer.cpp:612]619692 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_dump [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.418 [npu_driver.cc:4338] 619694 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.888.445 [tensor_data_transfer.cpp:612]619694 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_tensor_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.492 [npu_driver.cc:4338] 619698 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.888.517 [tensor_data_transfer.cpp:612]619698 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_scalar_summary [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.495 [npu_driver.cc:4338] 619700 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.888.549 [tensor_data_transfer.cpp:612]619700 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_histogram_summary [INFO] DEVICE(619443,fffeed3e8fa0,python3.9):2025-04-03-09:35:53.888.550 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619443,fffed5fbefa0,python3.9):2025-04-03-09:35:53.888.656 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619443,fffed7ffefa0,python3.9):2025-04-03-09:35:53.888.656 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619443,fffed4f9efa0,python3.9):2025-04-03-09:35:53.888.702 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.888.917 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is ms_histogram_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.943 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.888.956 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=5. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.889.403 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is ms_histogram_summary [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.889.438 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.889.461 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.889.504 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is ms_scalar_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.889.523 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 4. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.889.535 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=4. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.889.905 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is ms_scalar_summary [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.889.936 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.889.957 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [WARNING] DRV(619443,python3.9):2025-04-03-09:35:53.892.170 [queue_client_comm.c:114][ascend][curpid:619443,619696][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:53.892.285 [npu_driver.cc:4338] 619696 MemQueuePeek: queue empty [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.892.302 [tensor_data_transfer.cpp:612]619696 acltdtReceiveTensorV2: queue is empty, device is 6, name is ms_image_summary [INFO] DEVICE(619443,fffed6fdefa0,python3.9):2025-04-03-09:35:53.892.407 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.892.520 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is ms_image_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.892.543 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 3. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.892.557 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=3. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.892.962 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is ms_image_summary [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.892.996 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.893.019 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.893.068 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is ms_tensor_summary [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.893.088 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.893.099 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=2. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.893.487 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is ms_tensor_summary [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.893.516 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.893.537 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.893.574 [tensor_data_transfer.cpp:973]619443 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 6, name is ms_tensor_dump [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.893.593 [api_impl.cc:6065] 619443 MemQueueDestroy: Start to destroy queue on devId 6, qid is 1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.893.605 [npu_driver.cc:3989] 619443 MemQueueDestroy: deviceId=6, qid=1. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.893.960 [tensor_data_transfer.cpp:977]619443 acltdtDestroyChannel: acltdtDestroyChannel success, device is 6, name is ms_tensor_dump [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.893.984 [device.cpp:183]619443 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.894.004 [api_impl.cc:3419] 619443 DeviceReset: reset device, device_id=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.894.017 [runtime.cc:3444] 619443 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.894.034 [api_impl.cc:3449] 619443 DeviceReset: Succ, userDeviceId=6. [INFO] ASCENDCL(619443,python3.9):2025-04-03-09:35:53.894.045 [device.cpp:189]619443 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 6. [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.068 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.095 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.129 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.147 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.170 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_6 [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.188 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.209 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.226 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.251 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.389 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.429 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.448 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.467 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.492 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.519 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.542 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.560 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] PS(619443,ffff277eefa0,python3.9):2025-04-03-09:35:53.894.568 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] PS(619443,ffff26fdefa0,python3.9):2025-04-03-09:35:53.894.561 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619443,fffeecbd8fa0,python3.9):2025-04-03-09:35:53.894.582 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(619443,fffeb97aefa0,python3.9):2025-04-03-09:35:53.894.669 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.749 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.773 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.791 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.810 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.827 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.847 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.864 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.958 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.894.984 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.021 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.039 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.057 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.074 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.091 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.124 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.142 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.160 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.178 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.208 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.225 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.446 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.466 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.496 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.514 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.533 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.548 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619443,ffff964cc020,python3.9):2025-04-03-09:35:53.895.754 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:53.906.725 [awatchdog_monitor.c:160](tid:619475) thread 619711 not exist [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.964.743 [runtime.cc:5239] 619590 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:53.964.805 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:53.964.821 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619443,python3.9):2025-04-03-09:35:53.964.834 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619590] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.964.865 [runtime.cc:3077] 619590 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=6, devId=6, tdt status=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:53.965.007 [npu_driver.cc:551] 619590 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:54.319.923 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619443,ffff964cc020,python3.9):2025-04-03-09:35:54.320.069 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.036 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.483 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.517 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.868 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.889 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.906 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.923 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.944 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.670.959 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.671.866 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.671.889 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.014 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.051 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.417 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.440 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.456 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.469 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.484 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.495 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.508 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.521 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.536 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.548 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.560 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.574 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.589 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.600 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.613 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.625 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.638 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.653 [heartbeat.cc:145] [619443][DeInit]heartbeat deinit begin [INFO] HCCL(619443,python3.9):2025-04-03-09:35:54.672.664 [heartbeat.cc:183] [619443][DeInit]heartbeat deinit end [INFO] DEVICE(619443,ffff964cc020,python3.9):2025-04-03-09:35:54.673.000 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 6, group name: hccl_world_group [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:54.673.069 [hash_data.cpp:63] >>> (tid:619443) HashData uninitialize success [INFO] PROFILING(619443,python3.9):2025-04-03-09:35:54.673.096 [msprof_reporter_mgr.cpp:270] >>> (tid:619443) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.675.077 [api_impl.cc:2003] 619443 DevFree: device free mem=0x12c041200000 [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:54.676.460 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619443,ffff964cc020,python3.9):2025-04-03-09:35:54.676.497 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:54.676.659 [adx_dump_record.cpp:229][tid:619443] start to dump uninit [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:54.676.689 [adx_dump_record.cpp:239][tid:619443] dump uninit success [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:54.676.702 [adx_dump_record.cpp:674][tid:619676] exit record file thread [INFO] GE(619443,python3.9):2025-04-03-09:35:54.740.998 [operator_factory_impl.cc:392]619443 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.209 [operator_factory_impl.cc:392]619443 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.260 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.309 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.323 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.336 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.351 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.363 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.376 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.756.388 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.879.276 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.879.462 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] GE(619443,python3.9):2025-04-03-09:35:54.879.513 [opsproto_manager.cc:66]619443 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.936.609 [api_impl.cc:7151] 619443 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.936.881 [api_impl.cc:7135] 619443 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.938.932 [api_impl.cc:7151] 619443 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.939.098 [api_impl.cc:7135] 619443 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.940.813 [api_impl.cc:7121] 619443 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.944.449 [api_impl.cc:7121] 619443 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:54.950.918 [osal.cc:321] 619443 ThreadExit: ThreadExit start. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:55.176.732 [stars_engine.cc:1833] 619590 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:55.176.761 [stars_engine.cc:1836] 619590 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619443,ffff964cc020,python3.9):2025-04-03-09:35:55.177.000 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619443,python3.9):2025-04-03-09:35:55.216.927 [ops_kernel_builder_registry.cc:71]619443 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619443,python3.9):2025-04-03-09:35:55.216.982 [ops_kernel_builder_registry.cc:41]619443 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:56.516.184 [adx_server_manager.cpp:236][tid:619677] server manager stop [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:56.589.053 [adx_hdc_epoll.cpp:57][tid:619443] start to close hdc epoll [INFO] GE(619443,python3.9):2025-04-03-09:35:56.680.569 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0a86f530]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.682.464 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0a873b60]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.685.194 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0a89dfe0]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.689.950 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0a870b70]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.690.879 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0bcaa160]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.691.805 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0bcd2150]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.693.046 [op_impl_registry_holder_manager.cc:31]619443 CloseHandle:start close handle, handle[0xaaab0bd957d0]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.712.463 [mem_manager.cc:118]619443 Finalize:MemManager Finalize. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.712.503 [session_scope_mem_allocator.cc:73]619443 FreeAllMemory:Free all memory [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.061 [session_scope_mem_allocator.cc:73]619443 FreeAllMemory:Free all memory [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.089 [caching_allocator.cc:474]619443 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.126 [caching_allocator.cc:391]619443 FreeBlocks:Free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.144 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.155 [caching_allocator.cc:422]619443 FreeBlockBins:Free block bins. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.169 [caching_allocator.cc:474]619443 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.185 [caching_allocator.cc:391]619443 FreeBlocks:Free blocks. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.196 [caching_allocator.cc:358]619443 FreeCachedBlocks:Free cached blocks [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.208 [caching_allocator.cc:422]619443 FreeBlockBins:Free block bins. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.229 [mem_manager.cc:145]619443 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.248 [rts_caching_mem_allocator.cc:150]619443 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619443,python3.9):2025-04-03-09:35:56.715.304 [rts_caching_mem_allocator.cc:136]619443 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.749 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.813 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.826 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.842 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.853 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.869 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.880 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.894 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.908 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.919 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.931 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.942 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.954 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.969 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.980 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] HCCL(619443,python3.9):2025-04-03-09:35:56.805.991 [network_manager.cc:875] [619443]destroy all nic/vnic. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.811.818 [runtime.cc:1570] 619443 ~Runtime: deconstruct runtime [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.811.869 [runtime.cc:1577] 619443 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.172 [raw_device.cc:262] 619443 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.206 [engine.cc:188] 619443 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.234 [ringbuffer_maintain_task.cc:83] 619443 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.252 [stream.cc:1400] 619443 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.274 [stream.cc:2595] 619443 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=6 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.303 [npu_driver.cc:751] 619443 SqTaskSend: device_id=6, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.319 [stars_engine.cc:893] 619443 SendTask: Sq task send finished, device_id=6, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.338 [stream.cc:1494] 619443 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.355 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.372 [stream.cc:2886] 619443 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.391 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.406 [stream.cc:2886] 619443 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.421 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.435 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.451 [task.cc:185] 619443 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.468 [logger.cc:152] 619443 TaskFinished: device_id=6, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.493 [stars_engine.cc:437] 619443 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.509 [stream.cc:2886] 619443 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.525 [stars_engine.cc:524] 619443 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.537 [npu_driver.cc:5547] 619443 GetSqHead: deviceId=6, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.552 [stream.cc:852] 619443 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.566 [stream.cc:938] 619443 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.812.580 [stream.cc:1507] 619443 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.176 [stream_sqcq_manage.cc:451] 619443 FreeLogicCq: Return(0), threadIdentifier(281473203355680), devId(6), tsId(0), cqId(305), isFastCq(0). [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.222 [npu_driver.cc:6163] 619443 StreamUnBindLogicCq: success: deviceId=6, tsId=0, streamId=0, logicCq=305. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.306 [npu_driver.cc:1180] 619443 LogicCqFree: Succ:device_id=6,ts_id=0,type=2,logic_cq_id=305, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.326 [stream.cc:237] 619443 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.342 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.357 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.816.981 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.002 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.019 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.041 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=0, device_id=6, tsId=0, streamFlag=192 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.104 [stars_engine.cc:206] 619591 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.251 [stars_engine.cc:191] 619443 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.269 [osal.cc:225] 619443 Join: mmJoinTask failed, thread=281471557054368 does not exist! [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.283 [stars_engine.cc:150] 619443 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.306 [runtime.cc:5239] 619443 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=6, deviceId=6. [INFO] TDT(619443,python3.9):2025-04-03-09:35:56.817.327 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619443] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.344 [runtime.cc:3003] 619443 StopAicpuExecutor: TsdClose success, userDeviceId=6, devId=6, tdt status=0 [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:56.817.363 [atrace_api.c:118](tid:619443) AtraceDestroy start. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:56.817.405 [tracer_mgr_operate.c:248](tid:619443) destroy object RUNTIME_ATRACE_DEV6_TS0, exitSave(false). [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:56.817.424 [atrace_api.c:120](tid:619443) AtraceDestroy end. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.443 [stream.cc:237] 619443 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.459 [stream_sqcq_manage.cc:216] 619443 DeAllocStreamSqCq: deviceId=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.470 [npu_driver.cc:966] 619443 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.685 [npu_driver.cc:974] 619443 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=6, tsId=0, drvFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.704 [stream_sqcq_manage.cc:228] 619443 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.719 [npu_driver.cc:432] 619443 StreamIdFree: stream id free begin, device_id=6, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.817.737 [npu_driver.cc:441] 619443 StreamIdFree: Stream id free success, stream_id=1, device_id=6, tsId=0, streamFlag=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.827.160 [stars_engine.cc:58] 619443 ~StarsEngine: Destructor. [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.827.216 [engine.cc:84] 619443 ~Engine: Destructor. [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.072 [trs_dev_drv.c:370][ascend][curpid:619443,619443][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=6; fd=36; mmapFd=37) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.099 [devmm_svm.c:2238][ascend][curpid:619443,619443][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=6) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.119 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.141 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.184 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.208 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.224 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev6 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.243 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev6 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.258 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev6 Cached_size:1961984Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.274 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev6 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.293 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev6 Cached_size:2097152Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.308 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev6 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.327 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev6 Cached_size:2097152Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.343 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev6 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.362 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev6 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.381 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev6 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.408 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev6 Cached_size:23068672Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.422 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev6 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.441 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev6 Cached_size:6291456Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.455 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev6 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.473 [svm_mem_statistics.c:140][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev6 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.828.492 [svm_mem_statistics.c:146][ascend][curpid:619443,619443][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev6 Cached_size:16777216Bytes [INFO] DRV(619443,python3.9):2025-04-03-09:35:56.897.728 [drv_log_user.c:623][ascend][curpid:619443,619443][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=6; phyid=6; vfid=0; hostpid=619443; devpid=30773) [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.897.768 [npu_driver.cc:3045] 619443 DeviceClose: Close device success, device_id=6, tsId=0 [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:56.897.786 [raw_device.cc:125] 619443 ~Device: deconstruct device [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:57.006.140 [plugin_old_arch.cc:25] 619443 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:57.006.173 [runtime_keeper.cc:167] 619443 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619443,python3.9):2025-04-03-09:35:57.007.073 [task_fail_callback_data_manager.cc:63] 619443 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:57.105.951 [stacktrace_signal.c:188](tid:619443) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:57.157.560 [atrace_client_communication.c:141](tid:619443) send [end] msg. [INFO] DRV(619443,python3.9):2025-04-03-09:35:57.159.314 [hdc_core.c:1982][ascend][curpid:619443,619443][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4280; pid=619443) [INFO] IDEDD(619443,python3.9):2025-04-03-09:35:57.159.346 [hdc_api.cpp:671][tid:619443] connect succ, peer_node: 0, peer_devid: 6 [INFO] DRV(619443,python3.9):2025-04-03-09:35:57.159.421 [hdc_client.c:417][ascend][curpid:619443,619443][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4280) [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:57.159.476 [atrace_client_communication.c:164](tid:619443) send [end] msg success. [INFO] DRV(619443,python3.9):2025-04-03-09:35:58.222.231 [hdc_client.c:417][ascend][curpid:619443,619577][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4236) [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:58.222.292 [atrace_client_core.c:127](tid:619577) atrace receive thread exited, devId=6. [INFO] ATRACE(619443,python3.9):2025-04-03-09:35:58.222.801 [awatchdog_monitor.c:234](tid:619443) AwdMonitorExit [INFO] DRV(619443,python3.9):2025-04-03-09:35:58.289.277 [uda_user.c:147][ascend][curpid:619443,619443][drv][devmng][uda_char_dev_open]The fd and pid info: (fd=3; pid=619443) [INFO] DRV(619443,python3.9):2025-04-03-09:35:58.289.394 [uda_user.c:317][ascend][curpid:619443,619443][drv][devmng][_uda_init]User info. (admin_flag=1; local_flag=0; max_dev_num=100; max_udev_num=1124; uda_dev_num_davinci=8; uda_dev_num_kunpeng=0) output_allreduce/worker_4.log0000644000175100017500000667413414773362775017143 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:26.878.778 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:26.892.548 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 4 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.498 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.607 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.650 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.673 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.691 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.711 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.733 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.748 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.761 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.776 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.793 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.810 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.828 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.841 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.861 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.879 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.894 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.911 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.927 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.941 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.960 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.253.999 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.014 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.031 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.049 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.063 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.094 [ir_data_type_symbol_store.cc:240]619435 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.114 [ir_data_type_symbol_store.cc:240]619435 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.132 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.149 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.162 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.178 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.194 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.208 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.223 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.235 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.253 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.268 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.281 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.299 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.317 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.331 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.346 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.367 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.381 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.398 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.412 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.427 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.442 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.459 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.472 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.489 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.503 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.516 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.529 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.545 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.561 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.577 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.591 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.604 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.621 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.637 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.652 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.665 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.681 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.698 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.713 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.727 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.749 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.763 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.777 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.793 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.812 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.829 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.845 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.862 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.875 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.892 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.904 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.918 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.932 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.946 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.963 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.978 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.254.991 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.006 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.020 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.035 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.051 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.063 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.077 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.095 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.112 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.125 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.148 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.163 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.176 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.192 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.207 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.221 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.236 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.250 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.265 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.285 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.298 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.311 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.325 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.339 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.354 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.369 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.385 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.399 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.415 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.435 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.452 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.466 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.483 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.497 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.511 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.534 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.547 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.563 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.580 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.596 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.611 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.626 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.641 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.653 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.666 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.681 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.697 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.709 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.723 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.738 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.753 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.770 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.784 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.800 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.814 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.830 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.845 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.858 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.871 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.886 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.900 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.947 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.964 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.982 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.255.998 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.014 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.031 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.046 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.062 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.076 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.095 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.111 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.125 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.146 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.178 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.198 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.217 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.234 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.247 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.261 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.276 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.290 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.307 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.322 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.336 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.357 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.369 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.385 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.402 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.418 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.433 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.448 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.465 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.477 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.492 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.509 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.524 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.538 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.557 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.586 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.602 [ir_data_type_symbol_store.cc:173]619435 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.616 [ir_data_type_symbol_store.cc:219]619435 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619435,python3.9):2025-04-03-09:35:27.256.636 [ir_data_type_symbol_store.cc:182]619435 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.256 [awatchdog_core.c:99](tid:619435) watchdog get platform host side [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.411 [awatchdog_monitor.c:127](tid:619435) create monitor thread successfully, tid :281472722382752 [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.432 [awatchdog_monitor.c:226](tid:619435) AwdMonitorInit [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.612 [trace_attr.c:99](tid:619435) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.643 [trace_attr.c:189](tid:619435) attr init success, timeout=0ms. [WARNING] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.658 [trace_system_api.c:115](tid:619435) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.714 [trace_recorder.c:40](tid:619435) home_directory=/home/jenkins. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.732 [trace_recorder.c:121](tid:619435) use root path: /home/jenkins/ascend [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.746 [trace_event.c:151](tid:619435) init trace event success [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.263.799 [atrace_client_core.c:185](tid:619435) atrace client init success. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.102 [stacktrace_dumper.c:218](tid:619435) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.128 [trace_system_api.c:115](tid:619435) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.223 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 2 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.242 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 15 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.254 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 3 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.265 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 4 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.278 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 5 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.287 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 6 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.299 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 7 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.312 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 8 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.325 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 11 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.340 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 24 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.352 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 25 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.365 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 31 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.378 [stacktrace_signal.c:165](tid:619435) register signal handler for signo 35 succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.390 [stacktrace_signal.c:168](tid:619435) trace signal init succeed. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:27.265.402 [trace_core.c:51](tid:619435) trace init success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.266.079 [task_fail_callback_data_manager.cc:58] 619435 TaskFailCallBackManager: Constructor. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.275.731 [operator_factory_impl.cc:232]619435 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619435,python3.9):2025-04-03-09:35:27.275.765 [operator_factory_impl.cc:246]619435 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619435,python3.9):2025-04-03-09:35:27.275.779 [operator_factory_impl.cc:239]619435 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619435,python3.9):2025-04-03-09:35:27.282.970 [adapter_rts.cc:2720] [619435][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.299.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.299.282 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:27.299.309 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:27.310.362 [prof_cann_plugin.cpp:256] (tid:619435) Register module[6] callback. [INFO] OP(619435,python3.9):2025-04-03-09:35:27.361.179 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619435] cachelimit is 10000 [INFO] OP(619435,python3.9):2025-04-03-09:35:27.365.289 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619435] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619435,python3.9):2025-04-03-09:35:27.365.339 [op_dfx.cpp:54][NNOP][SystemConfig][619435] enable debug kernel flag 0. [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:27.365.357 [prof_cann_plugin.cpp:256] (tid:619435) Register module[61] callback. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.415.191 [mem_manager.cc:138]619435 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.421.696 [cache_policy.cc:47]619435 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.166 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.303 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.404 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.433 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.454 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.474 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.494 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.513 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.533 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.555 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.573 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.593 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.612 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.631 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.650 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.669 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.688 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.712 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.732 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.750 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.839 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.956 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.435.985 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.071 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.096 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.118 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.137 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.203 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.227 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.247 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.268 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.287 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.307 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.325 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.343 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.362 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.381 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.401 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.421 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.447 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.466 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.486 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.572 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.665 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.753 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.848 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.940 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.436.966 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.055 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.143 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.167 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.260 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.368 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.393 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.413 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.432 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.458 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.478 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.499 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.518 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.537 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.559 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.578 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.600 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.620 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.642 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.726 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.814 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.901 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.437.990 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.015 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.037 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.058 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.144 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.252 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.276 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.362 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.390 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.474 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.563 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.652 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.438.747 [addn.cc:29]619435 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.070 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.160 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.256 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.340 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.422 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.504 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.589 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.678 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.695 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.777 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.864 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619435,python3.9):2025-04-03-09:35:27.439.946 [symbolic_kernel_factory.cc:27]619435 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619435,python3.9):2025-04-03-09:35:27.446.068 [process_node_engine_manager.cc:140]619435 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619435,python3.9):2025-04-03-09:35:27.446.191 [process_node_engine_manager.cc:140]619435 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619435,python3.9):2025-04-03-09:35:27.452.406 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619435,python3.9):2025-04-03-09:35:27.452.450 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619435,python3.9):2025-04-03-09:35:27.452.980 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619435] start packet count timer [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:27.474.377 [prof_atls_plugin.cpp:117] (tid:619435) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:27.474.419 [prof_atls_plugin.cpp:117] (tid:619435) RegisterProfileCallback, callback type is 8 [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.540.891 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.541.342 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.582.551 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.582.678 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 4 [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.582.719 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.583.470 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.058 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.130 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.164 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.249 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.278 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.322 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.585.347 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.587.014 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.587.044 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.587.083 [acl.cpp:753]619435 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.587.140 [runtime_keeper.cc:78] 619435 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.716 [plugin_old_arch.cc:16] 619435 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaaf6a60c70 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.751 [runtime_keeper.cc:149] 619435 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.768 [runtime_keeper.cc:221] 619435 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.801 [runtime.cc:1387] 619435 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.827 [runtime.cc:1163] 619435 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.851 [npu_driver.cc:4062] 619435 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.868 [runtime.cc:5096] 619435 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.883 [runtime.cc:1173] 619435 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.895 [runtime.cc:1183] 619435 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.911 [runtime.cc:1193] 619435 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.925 [profiler.cc:84] 619435 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.941 [runtime.cc:1209] 619435 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.967 [runtime.cc:1219] 619435 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.980 [runtime.cc:1230] 619435 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.590.993 [runtime.cc:1242] 619435 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.006 [runtime.cc:1264] 619435 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.021 [runtime.cc:1270] 619435 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.136 [runtime.cc:109] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.532 [runtime.cc:121] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.614 [runtime.cc:1314] 619435 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.629 [runtime.cc:1322] 619435 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.744 [runtime.cc:1338] 619435 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.804 [utils.cc:105] 619435 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.856 [runtime.cc:1664] 619435 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.888 [runtime.cc:1688] 619435 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.906 [runtime.cc:1953] 619435 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.934 [runtime.cc:1477] 619435 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:27.591.953 [prof_atls_plugin.cpp:210] (tid:619435) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.970 [heterogenous.cc:19] 619435 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:27.591.999 [api_c_soc.cc:70] 619435 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.592.019 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.592.033 [acl.cpp:760]619435 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.592.065 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.628.151 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.661.913 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.662.362 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.662.794 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.663.592 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.664.042 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.665.803 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.665.854 [acl.cpp:753]619435 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.665.893 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:27.665.907 [acl.cpp:760]619435 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:27.665.933 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.592.719 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.594.519 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.614.490 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.623.213 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.626.156 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.629.153 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.656.968 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619435,python3.9):2025-04-03-09:35:28.671.469 [platform_info.cpp:853]619435 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.538.399 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.540.154 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.540.301 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.542.490 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.542.623 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.542.735 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.542.864 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.542.968 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.544.057 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.544.201 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.061 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.188 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.299 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.407 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.509 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619435,python3.9):2025-04-03-09:35:29.546.611 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619435,python3.9):2025-04-03-09:35:30.151.819 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619435,python3.9):2025-04-03-09:35:34.207.625 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.463.798 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.463.870 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.463.900 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619435,python3.9):2025-04-03-09:35:34.508.070 [ops_kernel_builder_registry.cc:55]619435 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619435,python3.9):2025-04-03-09:35:34.508.172 [ops_kernel_builder_registry.cc:28]619435 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.544.997 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.545.065 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.545.130 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.545.149 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:34.545.164 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619435,python3.9):2025-04-03-09:35:35.345.643 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:37.436.83 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:37.443.46 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:37.553.33 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.143.927 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:37.144.028 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.280 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.326 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.344 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.367 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.636 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471747665824 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.754 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471739211680 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.775 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.145.885 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471730757536 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.146.008 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471722303392 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.146.034 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.146.051 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.146.232 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58690, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.146.254 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [INFO] DISTRIBUTED(619435,ffff3e86afa0,python3.9):2025-04-03-09:35:37.146.296 [mindspore/ccsrc/distributed/rpc/tcp/connection.cc:239] ReconnectSourceSocket] Connection from 127.0.0.1:58690 to 127.0.0.1:8118 is not available yet, errno: 0 Success, events: 8220 [INFO] DISTRIBUTED(619435,ffff3e86afa0,python3.9):2025-04-03-09:35:37.146.346 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:35] DoDisconnect] Failed to call connect, fd: 20from : 127.0.0.1:58690 to: 127.0.0.1:8118, events: 8216, errno: 111 Connection refused [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:37.646.346 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:76] Connect] Failed to connect to the tcp server : 127.0.0.1:8118, retry to reconnect(1/1)... [INFO] DISTRIBUTED(619435,ffff3e86afa0,python3.9):2025-04-03-09:35:37.646.359 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 20 state is kDisconnecting. Start disconnecting from 127.0.0.1:58690 to 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.146.465 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:173] Register] Failed to connect to the meta server node url: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.146.491 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:363] ReconnectWithTimeoutWindow] Failed to register and try to reconnect to the meta server. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.646.565 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:386] Reconnect] Start reconnecting business client. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.646.584 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.646.723 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58776, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:38.646.741 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619435,ffff3f88afa0,python3.9):2025-04-03-09:35:38.646.765 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58776 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.817 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.836 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.849 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:393] Reconnect] Start reconnecting heartbeat client. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.873 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.966 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 22 source: 127.0.0.1:46846, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.146.983 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 3 [WARNING] DISTRIBUTED(619435,ffff3e86afa0,python3.9):2025-04-03-09:35:39.146.994 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:46846 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.056 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.074 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.089 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.103 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.250 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 4 has been registered successfully. [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:39.647.370 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619435,ffff3d84afa0,python3.9):2025-04-03-09:35:39.647.427 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619435,ffff3d84afa0,python3.9):2025-04-03-09:35:39.647.450 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.147.445 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.534 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3502.2 msec. [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.558 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.571 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.587 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 4 rank id: 4 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.601 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.620 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 9142 to 9397 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.638 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.064 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.667 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.687 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3502.44 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.708 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.803 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.820 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.850 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.877 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.892 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.647.906 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.649.599 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.649.619 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 9142 to 9397 [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.649.752 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:4812f2d1-1427-422a-816a-bfc61d2ed554, the ip:127.0.0.1, the port:9142 [INFO] PS(619435,ffff3d03afa0,python3.9):2025-04-03-09:35:40.649.774 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619435,ffff3d03afa0,python3.9):2025-04-03-09:35:40.649.797 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.651.366 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.651.491 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_4 has been registered successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.651.512 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:40.651.594 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.066 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 4 has been started successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.099 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.169 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.190 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.215 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 4, global rank size: 8 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.240 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3004.51 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.254 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 4 is devserver-c233-2 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.348 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.402 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.421 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 4 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.436 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 4 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.458 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [4]. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.478 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.22 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.539 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.566 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.585 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.652.610 [device.cpp:365]619435 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.652.703 [api_impl.cc:249] 619435 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.652.718 [api_impl.cc:6795] 619435 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.652.729 [device.cpp:371]619435 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.652.776 [callback.cpp:37]619435 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.652.811 [callback.cpp:44]619435 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.842 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.652.863 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.652.895 [device.cpp:143]619435 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.652.978 [npu_driver.cc:124] 619435 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.653.007 [api_impl.cc:2999] 619435 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.653.045 [runtime.cc:3602] 619435 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.653.273 [hdc_core.c:1982][ascend][curpid:619435,619435][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4245; pid=619435) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.653.408 [runtime.cc:3609] 619435 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.653.441 [atrace_client_communication.c:103](tid:619435) send [hello] msg. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.653.456 [atrace_client_communication.c:63](tid:619435) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.653.574 [hdc_core.c:1982][ascend][curpid:619435,619435][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4246; pid=619435) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:43.653.590 [hdc_api.cpp:671][tid:619435] connect succ, peer_node: 0, peer_devid: 4 [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.654.918 [hdc_client.c:417][ascend][curpid:619435,619435][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4246) [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.654.947 [atrace_client_communication.c:65](tid:619435) send msg by adx[finish], ret=0. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.654.959 [atrace_client_communication.c:97](tid:619435) msgType=1, magic=49959, version=1. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.654.971 [atrace_client_communication.c:135](tid:619435) send [hello] msg success. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.655.038 [atrace_client_core.c:160](tid:619435) start to receive device[4] trace log. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.655.068 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.655.083 [runtime.cc:2855] 619435 startAicpuExecutor: TsdOpen devId=4, device mode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.655.095 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.124 [client_manager.cpp:160][GetPlatformInfo][tid:619435] [TsdClient] mode[1] chipType[5] [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.655.157 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619435] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.168 [client_manager.cpp:582][GetVisibleDevices][tid:619435] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.181 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.655.178 [hdc_core.c:1982][ascend][curpid:619435,619581][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4247; pid=619435) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.193 [client_manager.cpp:125][GetInstance][tid:619435] [ClientManager] GetInstance, deviceId[4], g_runningMode[0], begin saving instance [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:43.655.200 [hdc_api.cpp:671][tid:619581] connect succ, peer_node: 0, peer_devid: 4 [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.228 [client_manager.cpp:435][IsHostEnvironment][tid:619435] [TsdClient] logicDeviceId is [4], hostaicpunum[0] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.243 [client_manager.cpp:464][GetClientRunMode][tid:619435] runningMode:0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.253 [client_manager.cpp:127][GetInstance][tid:619435] [ClientManager] Current mode:1 [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.655.264 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619435] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.275 [client_manager.cpp:347][GetProfilingMode][tid:619435] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.655.310 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619435] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.322 [process_mode_base.cpp:149][GetLogLevel][tid:619435] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.409 [process_mode_base.cpp:161][GetLogLevel][tid:619435] [TsdClient] Get Log Level[001] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.423 [process_mode_manager.cpp:71][OpenProcess][tid:619435] [ProcessModeManager] enter into open process deviceId[4] rankSize[0] [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.655.435 [atrace_client_communication.c:177](tid:619581) send [start] msg. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.436 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619435] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.655.483 [atrace_client_communication.c:189](tid:619581) send [start] msg success. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.498 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[.], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.567 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[scene.info], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.629 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.709 [client_manager.cpp:264][CheckPackageExists][tid:619435] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.723 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[ascend_install.info], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.782 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[version.info], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.840 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[..], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.655.898 [client_manager.cpp:261][CheckPackageExists][tid:619435] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.028 [client_manager.cpp:268][CheckPackageExists][tid:619435] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.042 [client_manager.cpp:283][CheckPackageExists][tid:619435] [TsdClient][deviceId=4] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.054 [client_manager.cpp:286][CheckPackageExists][tid:619435] [TsdClient][deviceId=4] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.656.130 [devdrv_hccl.c:103][ascend][curpid:619435,619435][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619435) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.145 [process_mode_manager.cpp:396][InitTsdClient][tid:619435] [TsdClient][deviceId=4] get process sign success, procPid[619435] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.174 [process_mode_manager.cpp:403][InitTsdClient][tid:619435] [TsdClient] deviceId[4] begin to init hdc client [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.207 [hdc_common.cpp:67][InitMsgSize][tid:619435] HdcCommon::InitMsgSize Start [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.220 [hdc_common.cpp:80][InitMsgSize][tid:619435] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.233 [hdc_client.cpp:184][CreateHdcSession][tid:619435] HdcClient::CreateHdcSession Start [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.310 [hdc_client.cpp:208][CreateHdcSession][tid:619435] [HdcClient] deviceId: 4 connect session [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.656.337 [hdc_core.c:1982][ascend][curpid:619435,619435][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4248; pid=619435) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.351 [hdc_client.cpp:219][CreateHdcSession][tid:619435] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.363 [hdc_client.cpp:230][CreateHdcSession][tid:619435] [HdcClient] deviceId: 4 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.377 [version_verify.cpp:36][SetVersionInfo][tid:619435] VersionVerify: send client version to server [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.415 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.435 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.656.448 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.656.476 [internal_api.cpp:101][GetScheduleEnv][tid:619435] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.013 [version_verify.cpp:68][PeerVersionCheck][tid:619435] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.029 [version_verify.cpp:89][ParseVersionInfo][tid:619435] VersionVerify: pass client version info success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.040 [hdc_client.cpp:275][CheckHdcConnection][tid:619435] Service[2] create hdc success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.052 [version_verify.cpp:122][SpecialFeatureCheck][tid:619435] VersionVerify: new type[35], supported [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.657.065 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619435] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.094 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619435] [TsdClient][deviceId=4] [sessionId=1] wait package info respond [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.657.108 [hdc_client.cpp:154][TsdRecvData][tid:619435] recv timeout is:150000 ms [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.151 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619435] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.166 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619435] [PackageInfoMsgProc] [sessionID=1] deviceId[4], checkCode[56684994] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.188 [hdc_client.cpp:172][TsdRecvData][tid:619435] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.202 [version_verify.cpp:122][SpecialFeatureCheck][tid:619435] VersionVerify: new type[64], supported [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.214 [hdc_client.cpp:411][Destroy][tid:619435] enter HdcClient::Destroy() function [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.658.227 [hdc_client.c:417][ascend][curpid:619435,619435][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4248) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.253 [hdc_client.cpp:394][DestroyClient][tid:619435] begin drvHdcClientDestroy [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.266 [hdc_client.cpp:399][DestroyClient][tid:619435] end drvHdcClientDestroy [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.277 [hdc_client.cpp:377][ClearClientPtr][tid:619435] begin HdcClient::ClearClientPtr [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.287 [hdc_client.cpp:385][ClearClientPtr][tid:619435] end HdcClient::ClearClientPtr [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.297 [hdc_client.cpp:418][Destroy][tid:619435] end HdcClient::Destroy() function [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.314 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619435] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.328 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619435] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.658.344 [devdrv_hccl.c:103][ascend][curpid:619435,619435][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619435) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.356 [process_mode_manager.cpp:396][InitTsdClient][tid:619435] [TsdClient][deviceId=4] get process sign success, procPid[619435] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.367 [process_mode_manager.cpp:403][InitTsdClient][tid:619435] [TsdClient] deviceId[4] begin to init hdc client [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.379 [hdc_common.cpp:67][InitMsgSize][tid:619435] HdcCommon::InitMsgSize Start [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.390 [hdc_common.cpp:80][InitMsgSize][tid:619435] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.403 [hdc_client.cpp:184][CreateHdcSession][tid:619435] HdcClient::CreateHdcSession Start [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.459 [hdc_client.cpp:208][CreateHdcSession][tid:619435] [HdcClient] deviceId: 4 connect session [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.658.485 [hdc_core.c:1982][ascend][curpid:619435,619435][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4249; pid=619435) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.498 [hdc_client.cpp:219][CreateHdcSession][tid:619435] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.508 [hdc_client.cpp:230][CreateHdcSession][tid:619435] [HdcClient] deviceId: 4 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.521 [version_verify.cpp:36][SetVersionInfo][tid:619435] VersionVerify: send client version to server [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.534 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.546 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.557 [version_verify.cpp:52][SetVersionInfo][tid:619435] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.825 [version_verify.cpp:68][PeerVersionCheck][tid:619435] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.841 [version_verify.cpp:89][ParseVersionInfo][tid:619435] VersionVerify: pass client version info success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.853 [hdc_client.cpp:275][CheckHdcConnection][tid:619435] Service[2] create hdc success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.866 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619435] [TsdClient] tsd get process sign successfully, procpid[619435] signSize[48] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.880 [version_verify.cpp:114][SpecialFeatureCheck][tid:619435] VersionVerify: previous type[6], supported [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.904 [process_mode_manager.cpp:88][OpenProcess][tid:619435] [ProcessModeManager] deviceId[4] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619435,python3.9):2025-04-03-09:35:43.658.918 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619435] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.658.929 [hdc_client.cpp:154][TsdRecvData][tid:619435] recv timeout is:150000 ms [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.871.504 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619435] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.871.536 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619435] [TsdClient] DeviceMsgProc recvMsg realDeviceId[4] msgType[8] localDevId[4] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.871.552 [hdc_client.cpp:172][TsdRecvData][tid:619435] [deviceId=4] the Tsd recv data already [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.871.586 [devmm_svm.c:132][ascend][curpid:619435,619435][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=4) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.873.769 [devmm_virt_interface.c:450][ascend][curpid:619435,619435][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.874.129 [drv_log_user.c:623][ascend][curpid:619435,619435][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=4; devid=4; vfid=0; hostpid=619435; devpid=30682) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.874.154 [devmm_svm.c:122][ascend][curpid:619435,619435][drv][devmm][devmm_set_device_info]Device info. (devid=4; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.874.439 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619435] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.874.458 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619435] enter into OpenInHost deviceid[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.874.497 [client_manager.cpp:435][IsHostEnvironment][tid:619435] [TsdClient] logicDeviceId is [4], hostaicpunum[0] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.874.512 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619435] host cpu not support [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.874.522 [process_mode_manager.cpp:121][OpenProcess][tid:619435] [TsdClient][deviceId=4] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.874.536 [runtime.cc:2879] 619435 startAicpuExecutor: userDeviceId=4, devId=4, tsdOpen ret=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.874.548 [runtime.cc:2891] 619435 startAicpuExecutor: TsdOpen success, devId=4, ret=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.874.562 [runtime.cc:3627] 619435 DeviceRetain: new dev ok id=4, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.874.582 [raw_device.cc:936] 619435 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.874.597 [npu_driver.cc:2968] 619435 DeviceOpen: Open device start, device_id=4, tsId=0. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.874.695 [trs_sqcq.c:458][ascend][curpid:619435,619435][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=4) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.875.156 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=6; num=2044) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.875.590 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=7; num=2044) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.875.825 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=10; num=1024) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.049 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=11; num=1024) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.508 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=12; num=2048) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.532 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=8; num=16) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.553 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=9; num=16) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.568 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=13; num=4) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.876.585 [trs_sqcq.c:394][ascend][curpid:619435,619435][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=4; tsId=0; id_type=14; num=10) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.878.713 [trs_dev_drv.c:302][ascend][curpid:619435,619435][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=4; fd=37; mmapFd=38) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.878.734 [devmm_svm.c:132][ascend][curpid:619435,619435][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=4) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.878.753 [devmm_svm.c:122][ascend][curpid:619435,619435][drv][devmm][devmm_set_device_info]Device info. (devid=4; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.878.827 [npu_driver.cc:3011] 619435 DeviceOpen: Open device success, device_id=4, tsId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.878.843 [raw_device.cc:292] 619435 Init: chipType:0x5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.878.888 [raw_device.cc:304] 619435 Init: isAddrFlat:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.878.904 [raw_device.cc:320] 619435 Init: GetDevInfo, device_id=4, hardwareVersion=0x503 [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.878.918 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.878.932 [npu_driver.cc:1578] 619435 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.879.882 [raw_device.cc:352] 619435 Init: Get phy info, phy chip id 4, phy die id 0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.879.901 [runtime.cc:2771] 619435 GetTsdQos: [TSD] start get qos, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.879.923 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.879.935 [tsd_client.cpp:189][TsdCapabilityGet][tid:619435] TsdCapabilityGet Begin. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.879.947 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.879.960 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.879.971 [process_mode_manager.cpp:1042][CapabilityGet][tid:619435] [ProcessModeManager] enter into CapabilityGet process deviceId[4] type[0] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.879.985 [process_mode_manager.cpp:386][InitTsdClient][tid:619435] [TsdClient] tsd client has been inited before [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.880.001 [version_verify.cpp:114][SpecialFeatureCheck][tid:619435] VersionVerify: previous type[38], supported [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.880.049 [process_mode_manager.cpp:1065][CapabilityGet][tid:619435] [TsdClient][deviceId=4] [sessionId=1] [type=0]send capability success [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.880.066 [hdc_client.cpp:154][TsdRecvData][tid:619435] recv timeout is:150000 ms [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.881.189 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619435] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.881.206 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619435] [TsdClient] PidQosMsgProc recvMsg realDeviceId[4] msgType[39] localDevId[4] rspCode[0],pidqos[1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.881.220 [hdc_client.cpp:172][TsdRecvData][tid:619435] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.881.232 [process_mode_manager.cpp:1070][CapabilityGet][tid:619435] [TsdClient][logicDeviceId_=4][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.246 [runtime.cc:2782] 619435 GetTsdQos: [TSD] get qos success, userDeviceId=4, devId=4, qos=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.262 [engine.cc:78] 619435 Engine: Constructor. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.273 [stars_engine.cc:51] 619435 StarsEngine: Constructor. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.282 [engine_factory.cc:26] 619435 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.294 [engine_factory.cc:38] 619435 CreateEngine: Engine init success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.303 [raw_device.cc:364] 619435 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.314 [spm_pool.cc:42] 619435 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.449 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.464 [spm_pool.cc:55] 619435 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.476 [raw_device.cc:380] 619435 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.881.489 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.503 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.881.512 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.881.531 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.888.915 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.888.946 [task_allocator.cc:46] 619435 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.888.978 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.888.993 [h2d_copy_mgr.cc:45] 619435 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.889.007 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.889.021 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.890.118 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.890.142 [task_allocator.cc:46] 619435 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.890.155 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.890.166 [h2d_copy_mgr.cc:45] 619435 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.890.176 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.890.192 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.033 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.053 [task_allocator.cc:46] 619435 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.063 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.078 [h2d_copy_mgr.cc:45] 619435 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.088 [uma_arg_loader.cc:86] 619435 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.103 [task_allocator.cc:46] 619435 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.126 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.138 [uma_arg_loader.cc:93] 619435 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.952 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.970 [uma_arg_loader.cc:102] 619435 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.891.983 [uma_arg_loader.cc:104] 619435 Init: ALLOC PCIE is support[1] [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.891.995 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.008 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.138 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.172 [h2d_copy_mgr.cc:45] 619435 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.185 [raw_device.cc:393] 619435 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.197 [raw_device.cc:398] 619435 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.209 [task_allocator.cc:46] 619435 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.369 [buffer_allocator.cc:61] 619435 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.553 [raw_device.cc:109] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.920 [raw_device.cc:121] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.936 [raw_device.cc:425] 619435 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.892.989 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.070 [stream.cc:2246] 619435 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.102 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.118 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.660 [npu_driver.cc:6125] 619435 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.743 [npu_driver.cc:6137] 619435 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=319, streamId=0, logicCq=319, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.771 [npu_driver.cc:6189] 619435 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=0, logicCq=319. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.792 [npu_driver.cc:5916] 619435 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e891f000, addr=dfffe891f000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.809 [stream.cc:653] 619435 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.866 [stream.cc:276] 619435 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.884 [stream.cc:688] 619435 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=4, streamResId=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.904 [stream_state_callback_manager.cc:45] 619435 Notify: not reg stream state callback func! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.920 [raw_device.cc:593] 619435 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.949 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.973 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=16 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.893.989 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.078 [tsch_stream.cc:45] 619435 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=4 streamResId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.097 [raw_device.cc:1219] 619435 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.204 [osal.cc:309] 619435 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.350 [osal.cc:182] 619435 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.374 [osal.cc:309] 619435 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.440 [osal.cc:182] 619435 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.447 [stars_engine.cc:1785] 619604 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.456 [stars_engine.cc:201] 619605 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.500 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.894.517 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.894.533 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.894.546 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.574 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.632 [npu_driver.cc:6306] 619604 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.654 [stars_engine.cc:1855] 619604 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.669 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.779 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.805 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.846 [task_info.cc:1596] 619435 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.861 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.876 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.910 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.925 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.894.942 [stars_engine.cc:1122] 619435 SyncTask: Task Wait: device_id=4, stream_id=0, task_id=0, logicCqId=319. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.002 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.042 [stars_engine.cc:1071] 619435 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.091 [stars_engine.cc:1017] 619435 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.128 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.157 [stars_engine.cc:1027] 619435 ProcReport: taskType=97. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.188 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.204 [stars_engine.cc:1570] 619435 ProcLogicCqReport: CQE warning, device_id=4, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.237 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.254 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.272 [task_info.cc:1614] 619435 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.287 [raw_device.cc:1305] 619435 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.321 [stars_engine.cc:1351] 619435 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.342 [npu_driver.cc:5618] 619435 GetSqEnable: deviceId=4, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.359 [stars_engine.cc:1365] 619435 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.374 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.388 [npu_driver.cc:5595] 619435 GetSqTail: deviceId=4, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.402 [npu_driver.cc:5406] 619435 SetSqHead: deviceId=4, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.421 [npu_driver.cc:5357] 619435 EnableSq: deviceId=4, tsId=0, sqId=0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.439 [stars_engine.cc:1430] 619435 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.458 [npu_driver.cc:5618] 619435 GetSqEnable: deviceId=4, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.472 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.488 [raw_device.cc:1331] 619435 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.895.502 [raw_device.cc:610] 619435 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.896.022 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.896.116 [ringbuffer_maintain_task.cc:98] 619435 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x17c0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.896.148 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.235 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.307 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.325 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.372 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.393 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.450 [stars_engine.cc:622] 619435 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=319, retCode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.467 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.480 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.492 [stream.cc:2886] 619435 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.507 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.548 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.614 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.630 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.648 [stars_engine.cc:437] 619435 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.903.664 [stream.cc:2886] 619435 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.063 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.088 [task_info.cc:111] 619435 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.105 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.118 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.137 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.150 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.163 [stars_engine.cc:1122] 619435 SyncTask: Task Wait: device_id=4, stream_id=0, task_id=2, logicCqId=319. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.312 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.330 [stars_engine.cc:1071] 619435 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.344 [stars_engine.cc:1017] 619435 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.360 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.374 [stars_engine.cc:1027] 619435 ProcReport: taskType=29. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.386 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.413 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.427 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.456 [npu_driver.cc:5618] 619435 GetSqEnable: deviceId=4, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.472 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.905.484 [raw_device.cc:1232] 619435 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.905.529 [atrace_api.c:40](tid:619435) AtraceCreate start. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.654 [trace_rb_log.c:45](tid:619435) init ring buffer time finished, realTime 1743644143905588340, monotonicTime 4211311553039, freq 99995 kHz, minutesWest 480 minutes [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.673 [trace_rb_log.c:137](tid:619435) [RUNTIME_ATRACE_DEV4_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.690 [tracer_mgr_operate.c:66](tid:619435) create object RUNTIME_ATRACE_DEV4_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.706 [atrace_api.c:44](tid:619435) AtraceCreate end. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.763 [trace_event.c:70](tid:619435) limitedNum 1, processType 0 [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.906.778 [trace_event.c:372](tid:619435) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.906.799 [runtime.cc:3159] 619435 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619435,python3.9):2025-04-03-09:35:43.906.818 [client_manager.cpp:196][SetProfilingCallback][tid:619435] [TsdClient] set profiling callback success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.906.872 [runtime.cc:3364] 619435 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.906.905 [context.cc:109] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.906.946 [context.cc:121] 619435 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.906.960 [context.cc:276] 619435 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.071 [raw_device.cc:803] 619435 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.088 [npu_driver_dcache_lock.cpp:39] 619435 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.267 [raw_device.cc:528] 619435 AllocStackPhyBase: AllocStackPhyBase device_id=4, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.319 [raw_device.cc:232] 619435 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.346 [runtime.cc:3393] 619435 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.363 [runtime.cc:3560] 619435 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.385 [api_impl.cc:2987] 619435 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.402 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.418 [api_impl.cc:2992] 619435 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:43.907.452 [awatchdog_core.c:175](tid:619435) create watchdog for id : 65543, tid : 619435, timeout : 300s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.907.468 [api_impl.cc:3025] 619435 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.907.509 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.907.527 [acl.cpp:121]619435 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.910.844 [api_impl.cc:3476] 619435 GetDeviceInfo: get device info, deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.910.879 [api_impl.cc:3476] 619435 GetDeviceInfo: get device info, deviceId=4 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.910.934 [device.cpp:133]619435 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 4, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.910.958 [device.cpp:154]619435 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.910.980 [context.cpp:87]619435 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.910.998 [context.cpp:98]619435 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.911.026 [stream.cpp:48]619435 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.067 [api_impl.cc:1120] 619435 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.127 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.166 [stream.cc:2246] 619435 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.215 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.233 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.731 [npu_driver.cc:6125] 619435 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.787 [npu_driver.cc:6137] 619435 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=320, streamId=2, logicCq=320, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.815 [npu_driver.cc:6189] 619435 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=2, logicCq=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.831 [npu_driver.cc:5916] 619435 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e00b3000, addr=dfffe00b3000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.846 [stream.cc:653] 619435 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.911.896 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.911.914 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.912.110 [stream.cc:291] 619435 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.915.975 [stream.cc:688] 619435 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=4, streamResId=65535 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.004 [stream_state_callback_manager.cc:45] 619435 Notify: not reg stream state callback func! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.035 [api_impl.cc:1162] 619435 StreamCreate: Succ, flags=1536, stream_id=2, context=aaab02fde2d0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.916.055 [stream.cpp:189]619435 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.077 [stream.cc:3161] 619435 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.916.093 [stream.cpp:195]619435 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.916.178 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab03f47eb0. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.916.248 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.916.265 [stream.cpp:48]619435 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.286 [api_impl.cc:1120] 619435 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.319 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.352 [stream.cc:2246] 619435 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.389 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.406 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.644 [npu_driver.cc:6125] 619435 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.694 [npu_driver.cc:6137] 619435 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=321, streamId=3, logicCq=321, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.717 [npu_driver.cc:6189] 619435 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=3, logicCq=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.734 [npu_driver.cc:5916] 619435 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0092000, addr=dfffe0092000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.750 [stream.cc:653] 619435 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619435,python3.9):2025-04-03-09:35:43.916.791 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.916.810 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.917.134 [stream.cc:291] 619435 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.917.791 [stream.cc:688] 619435 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=4, streamResId=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.917.811 [stream_state_callback_manager.cc:45] 619435 Notify: not reg stream state callback func! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.917.825 [api_impl.cc:1162] 619435 StreamCreate: Succ, flags=1536, stream_id=3, context=aaab02fde2d0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.917.842 [stream.cpp:189]619435 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.917.863 [stream.cc:3161] 619435 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.917.887 [stream.cpp:195]619435 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.917.926 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab04078dd0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.123 [callback.cpp:25]619435 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471351599008. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.150 [npu_driver.cc:462] 619435 EventIdAlloc: event id alloc begin, device_id=4, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.182 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.214 [npu_driver.cc:483] 619435 EventIdAlloc: Event id alloc success, device_id=4, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.228 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.231 [event.cc:347] 619435 GenEventId: Event id alloc success, device_id=4, tsId=0, eventFlag=1, event_id=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.247 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.276 [subscribe.cc:54] 619435 AssignGroupId: give groupId=320 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.293 [npu_driver.cc:3775] 619435 SqCqAllocate: deviceId=4, tsId=0, groupId=320. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.316 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.342 [npu_driver.cc:3785] 619435 SqCqAllocate: alloc success, deviceId=4 sqId=84, cqId=84. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.357 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.363 [subscribe.cc:131] 619435 Insert: callback subscribe: groupId=320, threadId=281471351599008, devId=4, tsId=0, streamId=2, sqId=84, cqId=84 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.380 [callback.cpp:31]619435 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471351599008. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.390 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.918.411 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab03f47eb0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.482 [callback.cpp:25]619435 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471343144864. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.504 [npu_driver.cc:462] 619435 EventIdAlloc: event id alloc begin, device_id=4, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.511 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.522 [npu_driver.cc:483] 619435 EventIdAlloc: Event id alloc success, device_id=4, ts_id=0, event_id=1, eventFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.545 [event.cc:347] 619435 GenEventId: Event id alloc success, device_id=4, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.545 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.560 [subscribe.cc:54] 619435 AssignGroupId: give groupId=321 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.579 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.586 [npu_driver.cc:3775] 619435 SqCqAllocate: deviceId=4, tsId=0, groupId=321. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.595 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.605 [npu_driver.cc:3785] 619435 SqCqAllocate: alloc success, deviceId=4 sqId=85, cqId=85. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.608 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.620 [subscribe.cc:131] 619435 Insert: callback subscribe: groupId=321, threadId=281471343144864, devId=4, tsId=0, streamId=3, sqId=85, cqId=85 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.918.635 [callback.cpp:31]619435 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471343144864. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.918.640 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.918.661 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab04078dd0. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.918.684 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.918.786 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.919.161 [npu_driver.cc:2616] 619435 MemGetInfoEx: memory info type=1, type=2, free=8863490048, hug_free=56245616640 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.230 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.325 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.370 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.397 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.419 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62092M, Reserved MOC size for Other Components(HCCL/rts/etc.):3890M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58202M. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.919.441 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.919.471 [event.cpp:260]619435 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.919.513 [api_impl.cc:5713] 619435 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.591 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.613 [timeout_set_task.cc:115] 619435 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.631 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.645 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.665 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.678 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.695 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.710 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.745 [stars_engine.cc:622] 619435 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=319, retCode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.764 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.777 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.790 [stream.cc:2886] 619435 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.805 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.818 [stream.cc:2886] 619435 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.835 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.848 [stream.cc:2886] 619435 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.861 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.876 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.889 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.902 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.921 [stars_engine.cc:437] 619435 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.937 [stream.cc:2886] 619435 StarsWaitForTask: No.4 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.920.953 [event.cpp:262]619435 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.920.967 [event.cpp:269]619435 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.920.988 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.014 [timeout_set_task.cc:115] 619435 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.031 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.045 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.063 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.076 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.093 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.107 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.122 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.135 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.150 [stream.cc:2886] 619435 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.164 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.176 [stream.cc:2886] 619435 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.189 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.202 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.216 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.229 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.246 [stars_engine.cc:437] 619435 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.260 [stream.cc:2886] 619435 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.921.273 [event.cpp:277]619435 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.303 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.379 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.401 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 4 has been set [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.420 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.437 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.492 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.523 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:43.921.586 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.921.734 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.756 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.921.791 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.921.830 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:43.921.848 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.921.861 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.922.019 [acl.cpp:288]619435 aclInit: start to execute aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.922.058 [acl.cpp:298]619435 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.922.176 [acl.cpp:242]619435 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.931.261 [error_manager.cc:673]619435 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:43.931.855 [acl.cpp:315]619435 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:43.931.936 [prof_atls_plugin.cpp:210] (tid:619435) Module[45] register callback of ctrl handle. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.931.986 [ge_executor.cc:273]619435 Initialize:Init GeExecutor begin. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.932.250 [op_lib_register.cc:117]619435 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.344 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.387 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.430 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.463 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.494 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.523 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.553 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.583 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.613 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.642 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.671 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.701 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.731 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.760 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.789 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.818 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.852 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.882 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.912 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.932.941 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.090 [op_lib_register.cc:165]619435 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.189 [plugin_manager.cc:616]619435 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.214 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.266 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.296 [plugin_manager.cc:624]619435 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619435,python3.9):2025-04-03-09:35:43.933.315 [plugin_manager.cc:206]619435 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619435,python3.9):2025-04-03-09:35:43.934.301 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.367 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.388 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.407 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.934.465 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.934.817 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.854 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.872 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.886 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.934.929 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.975 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.934.993 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.011 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.029 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.051 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.398 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.433 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.452 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.467 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.523 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.866 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.901 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.918 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.935.936 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.935.975 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.013 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.030 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.042 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:43.936.057 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.936.075 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.936.443 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.478 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.496 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.509 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.936.545 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.936.895 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.928 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.945 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.936.973 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.010 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.047 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.064 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.076 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.091 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.108 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.450 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.483 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.499 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.512 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.545 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.886 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.919 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.936 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.937.948 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.937.983 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.019 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.043 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.056 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.071 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.089 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.431 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.464 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.481 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.493 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.526 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.870 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.903 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.919 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.938.934 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.938.969 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.004 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.021 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.033 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.063 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.079 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.094 [plugin_manager.cc:218]619435 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.118 [plugin_manager.cc:422]619435 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.135 [plugin_manager.cc:155]619435 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.159 [plugin_manager.cc:158]619435 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.175 [plugin_manager.cc:426]619435 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.527 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.561 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.578 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.939.592 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.632 [plugin_manager.cc:439]619435 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.654 [plugin_manager.cc:175]619435 ReversePathString:Enter ReversePathString schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:43.939.705 [plugin_manager.cc:187]619435 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619435,python3.9):2025-04-03-09:35:43.940.047 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:43.940.082 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:43.940.105 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619435,python3.9):2025-04-03-09:35:43.940.178 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619435,python3.9):2025-04-03-09:35:43.956.762 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.957.584 [op_tiling_registry.cc:162]619435 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619435,python3.9):2025-04-03-09:35:43.957.732 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.957.875 [op_tiling_registry.cc:162]619435 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.024 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.157 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.280 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.414 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.434 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.456 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.580 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.697 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.811 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619435,python3.9):2025-04-03-09:35:43.958.933 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.070 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.494 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.607 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.740 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.863 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.959.980 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.079 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.196 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.378 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.478 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.579 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.678 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.697 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.801 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.910 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.960.929 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.028 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.129 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.225 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.418 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.588 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.683 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.781 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619435,python3.9):2025-04-03-09:35:43.961.895 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.311 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.418 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.525 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.701 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.810 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.962.927 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.024 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.122 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.227 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.325 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.434 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.533 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.551 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.644 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.745 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.846 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.865 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619435,python3.9):2025-04-03-09:35:43.963.958 [op_tiling_registry.cc:122]619435 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.060 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.171 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.272 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.381 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.591 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.680 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.964.930 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.011 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.089 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.236 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.293 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.422 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.498 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.574 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.648 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.731 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.819 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.848 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.926 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.955 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.965.981 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.141 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.228 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.310 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.390 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.466 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.546 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.698 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.776 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.966.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.134 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.237 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.324 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.415 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.493 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.967.573 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.971.897 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619435,python3.9):2025-04-03-09:35:43.971.930 [op_tiling_registry.cc:148]619435 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619435,python3.9):2025-04-03-09:35:43.973.476 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.973.613 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.973.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.033 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.292 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.425 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.579 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.974.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.017 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.137 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.383 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.514 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.643 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.975.944 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.227 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.357 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.505 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.632 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.976.921 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.308 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.422 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.549 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.675 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.918 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.977.946 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.978.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.978.307 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.978.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.978.672 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.978.797 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.180 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.296 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.325 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.462 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.490 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.609 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.722 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.871 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.979.994 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.109 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.242 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.459 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.486 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.596 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.714 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.980.920 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.033 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.156 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.394 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.603 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.766 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.981.926 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.351 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.469 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.580 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.705 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.982.982 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.152 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.213 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff09345d70, registered count:1 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.408 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff0934e248, registered count:2 [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.540 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.658 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.983.927 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.056 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.190 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.308 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.553 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.696 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.820 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.984.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.064 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.180 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.298 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.544 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.828 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.864 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.891 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.985.918 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.075 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.322 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.437 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.558 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.693 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.719 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.986.972 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.291 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.415 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.531 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.650 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.783 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.987.910 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.036 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.065 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.200 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.322 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.365 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.673 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.988.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.028 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.261 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.386 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.498 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.711 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.834 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.989.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.992.563 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.992.606 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.992.633 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.992.774 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.992.900 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.023 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.252 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.368 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.495 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.774 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.993.939 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.072 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.213 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.323 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.440 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.556 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.994.817 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.995.107 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.995.266 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.995.383 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.995.513 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.995.634 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.996.332 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.997.178 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.997.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.997.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.998.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.998.746 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.998.867 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.999.032 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.999.159 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:43.999.467 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.001.876 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.002.752 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.002.923 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.003.091 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.004.123 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.004.299 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.004.468 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.004.968 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.211 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.330 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.461 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.712 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.005.945 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.063 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.201 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.354 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.486 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.605 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.804 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.006.929 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.163 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.317 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.509 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.635 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.758 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.783 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.007.976 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.096 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.233 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.420 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.542 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.713 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.008.958 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.076 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.103 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.253 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.382 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.502 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.528 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.009.892 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.084 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.209 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.421 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.579 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.760 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.010.951 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.138 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.335 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.456 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.586 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.705 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.825 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.011.854 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.047 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.311 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.431 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.562 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.680 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.796 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.825 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.012.951 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.013.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.014.470 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.016.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.017.153 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.017.372 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.017.700 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.017.873 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.018.034 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.018.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.042 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.326 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.485 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.624 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.773 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.019.925 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.063 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.204 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.324 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.413 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.716 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.767 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.815 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.883 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.905 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.927 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.971 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.020.994 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.016 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.064 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.086 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.108 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.131 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.153 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.174 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.205 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.227 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.248 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.294 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.338 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.360 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.382 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.403 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.425 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.448 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.470 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.491 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.514 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.535 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.557 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.579 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.605 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.627 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.649 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.674 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.697 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.725 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.747 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.769 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.812 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.833 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.854 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.903 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.925 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.946 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.021.968 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.432 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.458 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.503 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.525 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.672 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.783 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.022.891 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.001 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.225 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.501 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.888 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.023.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.082 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.106 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.229 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.380 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.545 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.690 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.804 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.024.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.299 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.413 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.525 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.741 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.025.906 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.195 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.375 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.522 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.638 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.795 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.026.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.134 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.260 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.370 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.708 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.027.866 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.014 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.124 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.243 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.362 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.479 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.028.596 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.687 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.713 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.826 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.029.939 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.052 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.169 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.279 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.388 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.499 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.611 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.764 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.875 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.030.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.181 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.298 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.418 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.528 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.714 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.031.893 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.017 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.124 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.248 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.367 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.486 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.599 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.710 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.862 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.032.977 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.033.130 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.033.379 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.033.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.033.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.033.921 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.137 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.350 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.568 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.615 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.636 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.691 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.730 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.034.753 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.035.472 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.035.883 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.036.296 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.036.485 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.036.684 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.037.083 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.037.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.037.380 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.040.040 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.041.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.041.983 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.042.107 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.042.219 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.042.505 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.042.663 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.042.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.193 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.252 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.277 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.312 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.498 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.526 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.688 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.043.857 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.016 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.192 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.384 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.582 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.044.920 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.094 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.261 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.421 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.451 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.599 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.820 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.927 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619435,python3.9):2025-04-03-09:35:44.045.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.087 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.258 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.287 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.409 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.527 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.778 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.888 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.046.996 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.047.184 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.047.386 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.047.755 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.048.136 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.048.362 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.048.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.048.914 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.235 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.268 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.382 [op_tiling_registry.cc:134]619435 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.606 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.049.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.050.748 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.214 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.380 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.543 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.708 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.051.889 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.047 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.222 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.257 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.432 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.592 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.737 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.052.910 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.053.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.053.450 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.053.797 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.054.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.054.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.055.298 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.055.711 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.056.271 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.056.435 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.056.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.056.710 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.056.871 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.017 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.047 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.156 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.186 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.779 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.057.934 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.357 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.546 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.694 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.809 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.058.926 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.433 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.461 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.645 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.059.841 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.060.026 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.061.518 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.067.325 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.067.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.070.273 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.070.560 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.070.862 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.071.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.071.419 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.071.631 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.071.810 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.071.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.072.205 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.072.366 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.072.459 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.072.729 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.072.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.073.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.073.536 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.073.712 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.073.894 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.073.927 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.048 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.181 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.375 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.574 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.773 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.074.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.075.059 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.075.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.075.201 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.075.893 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.115 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.302 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.759 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.881 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.076.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.089 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.319 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.415 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.660 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.857 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.886 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.077.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.078.107 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.078.140 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.080.967 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.071 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.096 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.173 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.253 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.276 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.352 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.525 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.546 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.670 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.692 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.710 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.732 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.906 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.081.928 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.083.094 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff09c98a10, registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.083.130 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff09c98c70, registered count:1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.083.575 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.084.056 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.085.792 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.085.962 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.086.044 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.086.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.086.866 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.117.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.117.744 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.117.945 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.118.028 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.118.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.118.322 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.118.960 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.332 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.419 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.588 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.119.999 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.120.427 [op_tiling_manager.cc:79]619435 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.507 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.703 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.747 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.805 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.854 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.893 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.934 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.120.975 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.121.845 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.121.876 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.121.898 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.121.922 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.121.949 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.259 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff26e769b0, registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.281 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.303 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.321 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.336 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.635 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff26e79da0, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.658 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.675 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.691 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.707 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.122.979 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.123.036 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.123.122 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.123.827 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff26e302c0, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.123.855 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.123.872 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.123.900 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.123.916 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.245 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff26e34978, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.272 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.289 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.303 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.317 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.584 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.639 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.712 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.761 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.803 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.847 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.124.879 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.930 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.124.972 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.005 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.047 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.084 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.122 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.161 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.199 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.230 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.266 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.303 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.346 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.383 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.421 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.456 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.492 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.532 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.563 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.600 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.638 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.667 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.703 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.753 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.791 [op_tiling_manager.cc:73]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.831 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.868 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.125.901 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.939 [op_tiling_manager.cc:58]619435 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.125.978 [op_tiling_manager.cc:67]619435 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.010 [op_tiling_manager.cc:75]619435 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.032 [op_tiling_manager.cc:109]619435 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [192846] micro seconds. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.140 [plugin_manager.cc:639]619435 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.157 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.184 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.126.241 [host_cpu_engine.cc:254]619435 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.131.591 [host_cpu_engine.cc:273]619435 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619435,python3.9):2025-04-03-09:35:44.131.658 [host_cpu_engine.cc:254]619435 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.137.299 [host_cpu_engine.cc:267]619435 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619435,python3.9):2025-04-03-09:35:44.138.170 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619435] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.206 [folding.cc:404][InitCpuConstantFoldingNew][tid:619435]:"Init cpu constant folding begin." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.252 [folding.cc:408][InitCpuConstantFoldingNew][tid:619435]:"Number of registered ops: 234" [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.267 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Acos]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.286 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Acosh]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.300 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.314 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.327 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.342 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.353 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.365 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Add]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.379 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Addcdiv]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.391 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Addcmul]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.403 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[AffineGrid]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ApplyAdamV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.430 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Asin]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.443 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Asinh]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.455 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Assert]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.467 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Atan]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Atan2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.491 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Atanh]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.504 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[AvgPoolV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.517 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Betainc]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.544 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Bucketize]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.556 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CTCLossV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.568 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.580 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CacheSwapTable]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.592 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.608 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CaseCondition]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.620 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cast]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.630 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Ceil]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CheckNumerics]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.658 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ClipByValue]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.670 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ClipByValueV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.683 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Combinations]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.695 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CompareAndBitpack]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.709 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.722 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ConcatV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.733 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Conj]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.745 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ConjugateTranspose]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.761 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.773 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cosh]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.787 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[CropAndResize]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.799 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cross]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.811 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cummax]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.822 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cummin]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.835 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Cumsum]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.846 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Div]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.861 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[DynamicStitch]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.873 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EditDistance]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.887 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Elu]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.900 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EluGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.919 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.929 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.941 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.953 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.966 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.979 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.138.990 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.005 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.018 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.031 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.042 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.055 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.067 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.083 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.095 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingLookup]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.107 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.119 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.131 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.144 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.157 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.169 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Equal]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.182 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Expand]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.194 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ExpandDims]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.207 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Exponential]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.219 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.233 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Fill]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.246 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[FillDiagonal]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.261 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Fills]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.273 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Floor]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[FloorDiv]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.297 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[FloorMod]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.309 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GEMM]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.323 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GatherElements]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.334 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GatherV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.345 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Geometric]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.359 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GetDynamicDims]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.373 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Greater]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.389 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GreaterEqual]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.401 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GridSampler2D]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.429 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GridSampler3D]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.440 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.456 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Histogram]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.467 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Identity]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.492 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[IndexToAddr]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.504 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.517 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InitPartitionMap]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.529 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.542 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.553 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InvGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.566 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[InvertPermutation]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.578 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[IsClose]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.589 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[IsInf]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.600 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[IsNan]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.610 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Less]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.634 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[LinSpace]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.647 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[LinearSumAssignment]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.659 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Log]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.672 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Log1p]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.683 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.694 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.705 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.716 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MaskedFill]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.729 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MaskedScatter]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.742 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MaskedSelect]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.753 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.768 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MatMul]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.781 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.794 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MatrixDiagV3]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.806 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MinAreaPolygons]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.818 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Mul]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.828 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.840 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.852 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NLLLoss]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.877 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NLLLossGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.889 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NMSWithMask]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.903 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Neg]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.915 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.927 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NonZero]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.939 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.951 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.965 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[NotEqual]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.987 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[OnesLike]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.139.999 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[PSAMask]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.013 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[PSAMaskGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.025 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Pad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.038 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[PadD]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.050 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[PadV3]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.062 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[PadV3Grad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.074 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Pinverse]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.087 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Poisson]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.099 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Pow]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.112 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.126 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RNNTLoss]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.138 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RandomStandardNormal]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.149 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RandomUniform]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.175 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RealDiv]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.189 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ReduceAll]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.206 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ReduceMean]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.222 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ReduceSum]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.235 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Reshape]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.249 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Resize]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.260 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ResizeBilinear]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.273 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.290 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.302 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ResizeGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.316 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ResizeTrilinear]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.326 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ReverseSequence]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.339 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RightShift]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.353 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.366 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[RotatedNMS]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Round]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Rsqrt]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.409 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[STFT]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.426 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.440 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ScatterElements]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.452 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ScatterNdAdd]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.462 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ScatterNdMax]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.474 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ScatterNdMin]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.488 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SearchSorted]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.500 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SelectV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.513 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Sigmoid]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.527 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SigmoidGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.539 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Sign]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.554 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SilentCheck]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.564 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SilentCheckV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.575 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SilentCheckV3]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SliceWrite]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.598 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SoftmaxV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.610 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Sort]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.626 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.638 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SparseSegmentMean]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.648 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SparseSegmentSum]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.673 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SparseToDense]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.685 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SpatialTransformer]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.695 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SplitD]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.708 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SplitV]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.723 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Sqrt]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.738 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SqrtGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.751 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Square]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.764 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[SquaredDifference]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.776 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[StatelessBernoulli]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.790 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.802 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[StatelessRandperm]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.815 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[StridedSlice]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.827 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[StridedSliceV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.841 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Sub]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.851 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TableToResource]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.866 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Tan]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.880 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TanhGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.895 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TensorEqual]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.907 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.918 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ThreeNN]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.930 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Tile]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.945 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TileWithAxis]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.957 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TopK]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.970 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TopKPQDistance]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.986 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.140.997 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TopKV2D]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.011 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Trace]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.023 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TransData]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.036 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[TransDataRNN]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.050 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Tril]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.061 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Triu]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.075 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.086 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.098 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UninitPartitionMap]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.115 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UniqueConsecutive]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.128 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.140 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.155 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.167 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.181 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.193 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.207 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ViewCopy]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.219 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Voxelization]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.230 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[WarpAffine]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.241 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[ZerosLike]." [INFO] AICPU(619435,python3.9):2025-04-03-09:35:44.141.252 [folding.cc:413][InitCpuConstantFoldingNew][tid:619435]:"Register op[Zeta]." [INFO] OP(619435,python3.9):2025-04-03-09:35:44.141.270 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619435] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619435,python3.9):2025-04-03-09:35:44.141.284 [host_cpu_engine.cc:273]619435 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619435,python3.9):2025-04-03-09:35:44.141.407 [plugin_manager.cc:872]619435 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.144.657 [plugin_manager.cc:883]619435 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.733 [ops_kernel_executor_manager.cc:37]619435 Initialize:No library to load [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.750 [plugin_manager.cc:540]619435 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.761 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.792 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.813 [plugin_manager.cc:547]619435 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.144.828 [plugin_manager.cc:206]619435 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.145.430 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.480 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.496 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.509 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.145.555 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.145.873 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.909 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.925 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.145.939 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.145.975 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.015 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.031 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.044 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.058 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.087 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.102 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.115 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.144 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.466 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.499 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.515 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.526 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.561 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.880 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.914 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.929 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.146.940 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.146.972 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.006 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.019 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.034 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.047 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.076 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.089 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.102 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.126 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.443 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.478 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.493 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.504 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.545 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.867 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.900 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.915 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.927 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.147.958 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.147.990 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.004 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.014 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.028 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.057 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.071 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.092 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.107 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.427 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.462 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.477 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.488 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.520 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.838 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.870 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.885 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.896 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.148.927 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.963 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.976 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.148.989 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.002 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.029 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.043 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.066 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.079 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.396 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.429 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.442 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.454 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.487 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.809 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.839 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.854 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.866 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.898 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.930 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.943 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.957 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.149.969 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.149.996 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.150.008 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.150.030 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.041 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.050 [plugin_manager.cc:218]619435 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.071 [plugin_manager.cc:422]619435 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.081 [plugin_manager.cc:155]619435 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.100 [plugin_manager.cc:158]619435 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.111 [plugin_manager.cc:426]619435 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.420 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.150.454 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.150.467 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.150.479 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.512 [plugin_manager.cc:439]619435 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.544 [ge_executor.cc:259]619435 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.891 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.923 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:44.150.942 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.043 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.059 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.132 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.145 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.176 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.222 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.238 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.269 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.312 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.325 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.387 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.412 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.474 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.486 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.515 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.556 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.568 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.595 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.634 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.647 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.708 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.720 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.785 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.798 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.827 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.869 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.885 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.151.911 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.963 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.151.977 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.040 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.054 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.117 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.131 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.168 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.217 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.231 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.261 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.302 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.318 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.380 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.396 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.459 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.473 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.503 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.543 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.565 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.595 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.636 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.649 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.678 [opsproto_manager.cc:123]619435 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.717 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.783 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.798 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.812 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.152.826 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.152.840 [opsproto_manager.cc:147]619435 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.153.250 [operator_factory_impl.cc:215]619435 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619435,python3.9):2025-04-03-09:35:44.153.275 [operator_factory_impl.cc:269]619435 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619435,python3.9):2025-04-03-09:35:44.153.307 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.158.764 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.527 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.669 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.718 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.752 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.797 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.828 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.856 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.884 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.159.912 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.180.133 [operator_factory_impl.cc:368]619435 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619435,python3.9):2025-04-03-09:35:44.180.385 [operator_factory_impl.cc:284]619435 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.560 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.583 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.600 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.614 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.627 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619435,python3.9):2025-04-03-09:35:44.180.648 [operator_factory_impl.cc:254]619435 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619435,python3.9):2025-04-03-09:35:44.180.671 [operator_factory_impl.cc:340]619435 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619435,python3.9):2025-04-03-09:35:44.180.694 [operator_factory_impl.cc:307]619435 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.726 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.744 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.760 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.180.867 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.662 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.699 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.729 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.746 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.763 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.775 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.793 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.831 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.850 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.864 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.882 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.903 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.918 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.946 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.975 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.181.994 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.061 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.100 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.122 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.137 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.163 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.182.186 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.312 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.337 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.356 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.375 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.388 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.403 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.432 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.472 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.485 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.500 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.589 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.606 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.622 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.638 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.651 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.662 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.964 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.984 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.183.998 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.012 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.022 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.032 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.046 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.068 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.084 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.097 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.111 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.129 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.141 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.381 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.405 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.471 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.487 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.500 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.513 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.184.523 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.386 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.413 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.430 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.451 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.467 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.480 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.492 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.611 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.185.695 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.040 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.061 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.074 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.086 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.098 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.113 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.562 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.871 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.894 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.908 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.186.924 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.092 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.239 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.520 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.545 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.586 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.187.600 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.154 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.217 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.235 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.259 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.278 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.300 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.321 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.340 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.392 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.432 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.188.821 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.189.019 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.189.266 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.189.299 [operator_factory_impl.cc:222]619435 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619435,python3.9):2025-04-03-09:35:44.189.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.189.942 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.189.967 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.189.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.054 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.081 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.127 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.149 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.172 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.196 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.218 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.238 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.263 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.288 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.309 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.328 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.349 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.371 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.391 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.411 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.431 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.449 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.472 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.495 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.518 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.544 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.574 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.601 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.633 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.675 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.698 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.719 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.764 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.788 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.809 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.875 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.924 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.970 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.190.991 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.012 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.037 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.059 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.081 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.101 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.148 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.171 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.202 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.249 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.276 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.305 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.329 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.351 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.372 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.393 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.418 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.439 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.461 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.484 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.505 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.529 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.553 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.603 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.626 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.648 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.671 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.719 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.742 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.765 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.791 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.818 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.842 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.863 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.886 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.938 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.962 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.191.983 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.009 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.033 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.080 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.142 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.175 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.198 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.222 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.246 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.295 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.318 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.341 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.363 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.396 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.419 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.441 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.463 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.487 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.535 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.557 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.582 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.605 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.627 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.647 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.675 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.698 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.718 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.768 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.788 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.806 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.833 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.856 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.874 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.893 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.914 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.937 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.192.989 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.013 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.036 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.059 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.083 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.103 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.125 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.144 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.164 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.203 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.222 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.242 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.261 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.375 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.402 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.426 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.450 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.474 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.497 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.543 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.568 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.647 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.692 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.716 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.738 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.764 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.789 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.811 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.835 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.884 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.908 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.931 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.954 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.193.977 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.000 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.051 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.076 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.125 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.149 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.198 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.278 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.305 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.334 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.361 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.385 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.409 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.433 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.483 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.536 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.561 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.582 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.627 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.652 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.675 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.700 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.747 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.770 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.815 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.846 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.870 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.891 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.935 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.956 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.194.983 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.007 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.031 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.083 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.103 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.126 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.148 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.171 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.193 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.297 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.395 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.494 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.591 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.718 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.774 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.195.915 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.013 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.114 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.330 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.444 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.557 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.629 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.724 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.830 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.855 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.876 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.899 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.196.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.068 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.174 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.284 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.392 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.417 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.528 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.629 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.691 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.789 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.889 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.197.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.084 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.107 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.200 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.298 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.324 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.522 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.620 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.843 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.198.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.044 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.238 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.344 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.372 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.422 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.442 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.465 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.488 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.537 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.563 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.585 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.710 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.812 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.838 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.877 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.896 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.199.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.110 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.224 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.259 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.359 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.465 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.492 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.512 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.615 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.718 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.830 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.200.932 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.032 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.138 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.234 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.331 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.529 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.637 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.662 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.684 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.779 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.877 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.201.991 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.301 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.395 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.417 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.512 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.610 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.801 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.824 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.202.924 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.024 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.118 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.214 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.315 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.419 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.444 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.465 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.554 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.577 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.595 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.613 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.631 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.650 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.682 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.700 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.718 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.737 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.754 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.810 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.847 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.203.960 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.054 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.153 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.266 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.366 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.468 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.628 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.653 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.751 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.777 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.819 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.842 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.204.982 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.001 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.296 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.483 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.578 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.680 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.803 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.205.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.017 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.114 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.220 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.346 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.374 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.491 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.597 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.701 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.798 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.899 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.206.994 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.092 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.198 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.309 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.516 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.613 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.717 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.940 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.207.983 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.006 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.026 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.051 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.070 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.201 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.230 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.338 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.432 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.530 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.630 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.737 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.937 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.208.959 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.057 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.153 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.250 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.360 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.455 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.652 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.679 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.702 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.723 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.768 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.794 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.816 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.838 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.890 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.914 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.957 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.209.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.000 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.022 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.047 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.068 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.159 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.267 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.292 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.483 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.582 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.758 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.863 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.210.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.033 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.128 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.223 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.532 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.632 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.732 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.831 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.211.943 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.047 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.250 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.351 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.373 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.467 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.665 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.756 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.848 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.212.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.049 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.147 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.240 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.336 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.542 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.640 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.662 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.762 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.858 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.213.956 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.070 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.277 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.377 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.474 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.768 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.868 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.889 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.214.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.081 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.178 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.269 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.375 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.472 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.640 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.215.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.152 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.366 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.462 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.692 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.888 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.931 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.950 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.970 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.216.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.006 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.024 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.128 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.151 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.178 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.275 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.368 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.470 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.674 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.702 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.723 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.817 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.924 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.217.946 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.040 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.133 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.236 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.333 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.356 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.377 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.497 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.541 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.631 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.666 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.784 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.806 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.848 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.218.940 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.038 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.134 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.157 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.281 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.303 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.394 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.491 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.594 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.637 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.730 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.753 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.775 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.876 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.219.984 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.192 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.294 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.496 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.594 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.698 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.802 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.220.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.002 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.301 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.402 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.426 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.445 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.542 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.585 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.684 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.881 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.221.993 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.105 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.309 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.406 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.503 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.602 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.625 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.222.896 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.002 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.105 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.306 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.410 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.508 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.701 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.797 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.903 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.223.999 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.097 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.225 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.321 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.418 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.623 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.722 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.818 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.224.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.016 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.109 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.204 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.299 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.422 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.624 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.225.928 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.038 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.241 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.341 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.445 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.545 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.648 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.744 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.844 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.226.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.147 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.242 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.340 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.435 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.541 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.636 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.227.834 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.031 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.426 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.545 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.693 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.727 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.759 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.805 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.868 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.893 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.918 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.948 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.968 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.228.985 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.004 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.024 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.056 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.078 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.273 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.372 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.468 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.567 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.774 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.890 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.229.990 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.100 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.205 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.310 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.412 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.536 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.558 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.578 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.596 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.725 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.758 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.785 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.820 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.852 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.883 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.230.971 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.001 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.028 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.065 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.128 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.162 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.190 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.216 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.239 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.445 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.548 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.647 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.848 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.231.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.077 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.185 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.283 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.382 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.483 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.507 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.613 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.803 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.232.980 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.058 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.169 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.274 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.370 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.474 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.650 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.233.926 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.023 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.122 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.216 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.324 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.458 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.557 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.762 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.234.969 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.096 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.299 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.401 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.511 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.640 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.674 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.700 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.724 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.748 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.801 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.895 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.235.999 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.072 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.128 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.244 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.452 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.761 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.785 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.807 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.947 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.236.970 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.043 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.140 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.340 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.437 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.537 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.638 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.738 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.836 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.237.944 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.036 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.144 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.246 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.441 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.648 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.733 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.760 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.238.983 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.198 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.308 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.415 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.517 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.544 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.569 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.615 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.641 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.726 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.749 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.770 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.792 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.813 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.239.917 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.256 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.289 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.315 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.412 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.521 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.620 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.731 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.890 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.240.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.136 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.241 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.351 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.379 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.406 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.509 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.540 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.641 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.739 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.838 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.241.951 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.062 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.087 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.183 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.206 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.307 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.334 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.358 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.382 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.405 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.430 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.454 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.479 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.501 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.523 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.546 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.573 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.598 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.623 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.647 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.746 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.943 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.966 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.242.996 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.019 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.064 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.087 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.111 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.243.137 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.189 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.297 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.384 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.459 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.539 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.612 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.730 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.244.925 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.002 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.159 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.195 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.323 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.396 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.516 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.593 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.666 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.245.741 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.058 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.084 [opsproto_manager.cc:162]619435 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.151 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.200 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.730 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.790 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.811 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.905 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.920 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.982 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.246.996 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.007 [opp_so_manager.cc:56]619435 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.020 [plugin_manager.cc:540]619435 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.031 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.054 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.072 [plugin_manager.cc:547]619435 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.084 [plugin_manager.cc:206]619435 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.456 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.496 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.513 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.525 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.568 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.247.910 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.945 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.961 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.247.974 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.011 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.055 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.073 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.089 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.100 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.145 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.175 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.190 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.202 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.529 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.567 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.582 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.248.594 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.626 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.248.966 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.000 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.015 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.027 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.060 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.101 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.115 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.129 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.143 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.189 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.203 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.213 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.225 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.546 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.583 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.601 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.249.614 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.654 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.249.989 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.022 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.037 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.053 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.089 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.123 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.135 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.147 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.159 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.203 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.218 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.231 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.245 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.563 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.596 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.611 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.250.626 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.662 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.250.982 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.014 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.029 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.042 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.075 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.107 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.120 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.133 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.166 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.195 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.208 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.221 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.234 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.550 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.584 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.598 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.251.611 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.644 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.251.970 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.006 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.020 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.033 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.066 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.100 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.114 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.127 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.154 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.195 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.210 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.220 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.233 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.243 [plugin_manager.cc:218]619435 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.266 [plugin_manager.cc:422]619435 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.278 [plugin_manager.cc:155]619435 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.300 [plugin_manager.cc:158]619435 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.314 [plugin_manager.cc:426]619435 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.623 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.655 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.671 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.252.683 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.252.716 [plugin_manager.cc:439]619435 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.028 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.057 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.077 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.185 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.203 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.253.216 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.255.613 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.255.663 [op_impl_registry_holder_manager.cc:216]619435 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.255.680 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.255.726 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.256.253 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.256.381 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.256.401 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.256.417 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.258.622 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.258.666 [op_impl_registry_holder_manager.cc:216]619435 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.258.697 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.258.735 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.092 [op_impl_space_registry.cc:80]619435 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.115 [op_impl_space_registry.cc:83]619435 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.134 [op_impl_space_registry.cc:80]619435 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.146 [op_impl_space_registry.cc:83]619435 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619435,python3.9):2025-04-03-09:35:44.259.157 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.208 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.259.247 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.259.337 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.259.353 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.259.367 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.260.206 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.260.241 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.260.344 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.260.360 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.260.374 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.261.069 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.261.110 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.261.156 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.261.191 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.261.274 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.261.290 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.261.301 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.261.959 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.261.985 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.077 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.094 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.104 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.735 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.262.762 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.262.806 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.262.838 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.939 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.956 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.262.968 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.263.551 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.263.577 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.263.667 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.263.683 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.263.693 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.264.391 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.264.419 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.264.454 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.264.486 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.264.564 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.264.578 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.264.588 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.265.248 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.265.276 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619435,python3.9):2025-04-03-09:35:44.265.375 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.265.391 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.265.403 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.265.983 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.266.010 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.266.046 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.266.079 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.266.119 [opp_so_manager.cc:97]619435 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.266.195 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.266.239 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.266.255 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.266.267 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.799 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.880 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.896 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.915 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.931 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.947 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.299.993 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.010 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.024 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.071 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.089 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.135 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.150 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.225 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.243 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.258 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.275 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.288 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.303 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.319 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.334 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.352 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.367 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.385 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.400 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.434 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.477 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.492 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.525 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.540 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.556 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.572 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.589 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.603 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.619 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.635 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.649 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.711 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.726 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.759 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.776 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.791 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.806 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.823 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.838 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.857 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.877 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.896 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.926 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.942 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.960 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.300.993 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.009 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.058 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.074 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.091 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.105 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.118 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.135 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.149 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.167 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.199 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.218 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.234 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.248 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.263 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.278 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.293 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.313 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.328 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.343 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.358 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.374 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.404 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.420 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.436 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.450 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.465 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.480 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.499 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.515 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.530 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.546 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.561 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.577 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.594 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.609 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.624 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.639 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.684 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.705 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.722 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.744 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.761 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.778 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.799 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.815 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.849 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.883 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.899 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.939 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.956 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.973 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.301.989 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.006 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.023 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.044 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.061 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.079 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.101 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.118 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.135 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.152 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.169 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.185 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.206 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.231 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.249 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.376 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.417 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.435 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.452 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.469 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.486 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.503 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.539 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.556 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.572 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.623 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.643 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.660 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.676 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.693 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.710 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.727 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.770 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.792 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.810 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.844 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.861 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.878 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.895 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.932 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.970 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.302.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.006 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.023 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.040 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.058 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.074 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.091 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.108 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.124 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.144 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.160 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.194 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.211 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.228 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.252 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.269 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.287 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.305 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.325 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.343 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.360 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.376 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.393 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.410 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.444 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.464 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.498 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.515 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.532 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.566 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.599 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.619 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.636 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.652 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.687 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.709 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.816 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.303.920 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.138 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.288 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.310 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.330 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.431 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.531 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.641 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.741 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.844 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.304.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.075 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.098 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.117 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.216 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.323 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.348 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.368 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.405 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.425 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.451 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.550 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.648 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.755 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.865 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.887 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.305.995 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.096 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.138 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.157 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.255 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.459 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.555 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.784 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.809 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.306.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.018 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.231 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.255 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.451 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.560 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.668 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.765 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.874 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.895 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.915 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.933 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.971 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.307.989 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.009 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.027 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.062 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.079 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.188 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.298 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.321 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.340 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.357 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.376 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.393 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.494 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.592 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.833 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.962 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.308.984 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.077 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.281 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.383 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.490 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.597 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.700 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.805 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.309.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.014 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.118 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.141 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.161 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.360 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.383 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.482 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.585 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.690 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.892 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.310.919 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.031 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.142 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.239 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.365 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.469 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.671 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.781 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.311.989 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.012 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.034 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.134 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.168 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.194 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.213 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.232 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.250 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.268 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.286 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.303 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.320 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.337 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.356 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.376 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.406 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.445 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.465 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.572 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.676 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.775 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.878 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.312.982 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.084 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.189 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.211 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.257 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.365 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.388 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.409 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.427 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.527 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.551 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.588 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.685 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.788 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.313.889 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.009 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.111 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.210 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.310 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.525 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.633 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.731 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.833 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.937 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.314.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.009 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.114 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.222 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.319 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.536 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.640 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.851 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.315.962 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.062 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.284 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.400 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.532 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.637 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.660 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.687 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.708 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.728 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.749 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.768 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.789 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.886 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.316.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.012 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.113 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.221 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.327 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.436 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.538 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.643 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.771 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.881 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.317.980 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.083 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.302 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.412 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.433 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.454 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.471 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.488 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.510 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.527 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.544 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.584 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.626 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.650 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.668 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.685 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.703 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.721 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.738 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.755 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.954 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.318.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.077 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.179 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.285 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.364 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.466 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.573 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.755 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.855 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.319.958 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.063 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.175 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.276 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.380 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.484 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.591 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.615 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.718 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.817 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.320.917 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.017 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.143 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.249 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.456 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.559 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.659 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.782 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.886 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.321.990 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.090 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.303 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.404 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.511 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.535 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.631 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.732 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.322.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.062 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.166 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.266 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.371 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.479 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.580 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.684 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.323.910 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.232 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.346 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.451 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.558 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.668 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.769 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.868 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.324.976 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.076 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.184 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.291 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.393 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.497 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.725 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.854 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.873 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.892 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.929 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.947 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.325.984 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.080 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.110 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.131 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.232 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.347 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.449 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.551 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.659 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.701 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.798 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.905 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.326.927 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.127 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.228 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.334 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.359 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.385 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.406 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.511 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.535 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.554 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.652 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.672 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.691 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.797 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.819 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.841 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.327.961 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.073 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.186 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.213 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.233 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.331 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.355 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.455 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.562 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.666 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.689 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.711 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.817 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.839 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.860 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.328.960 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.063 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.169 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.272 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.380 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.490 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.329.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.027 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.126 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.230 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.339 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.440 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.552 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.578 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.600 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.696 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.716 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.735 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.836 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.330.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.148 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.359 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.462 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.561 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.663 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.766 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.811 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.331.944 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.046 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.151 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.263 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.366 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.479 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.582 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.789 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.332.887 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.013 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.117 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.225 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.325 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.425 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.531 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.637 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.846 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.333.944 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.044 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.156 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.259 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.367 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.478 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.580 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.601 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.701 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.810 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.334.918 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.019 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.122 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.230 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.333 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.450 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.551 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.657 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.757 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.855 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.335.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.069 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.288 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.392 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.496 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.709 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.809 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.336.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.026 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.136 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.241 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.345 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.446 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.572 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.597 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.621 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.690 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.713 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.734 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.754 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.775 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.792 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.809 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.847 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.867 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.337.974 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.184 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.285 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.388 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.496 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.603 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.692 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.717 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.824 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.338.924 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.025 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.131 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.236 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.368 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.405 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.423 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.547 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.573 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.598 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.620 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.643 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.693 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.714 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.735 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.756 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.776 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.828 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.854 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.908 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.930 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.339.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.021 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.125 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.146 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.373 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.580 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.678 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.784 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.340.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.014 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.119 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.217 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.238 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.335 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.434 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.541 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.650 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.730 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.812 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.341.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.130 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.234 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.344 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.423 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.505 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.585 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.803 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.909 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.342.992 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.099 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.122 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.218 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.315 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.524 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.627 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.728 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.343.932 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.049 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.151 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.265 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.386 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.407 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.428 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.448 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.466 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.511 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.611 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.715 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.812 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.344.933 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.036 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.135 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.238 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.346 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.447 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.469 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.489 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.624 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.649 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.721 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.345.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.146 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.253 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.357 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.563 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.634 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.655 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.753 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.863 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.346.962 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.068 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.171 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.272 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.374 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.451 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.476 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.579 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.785 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.890 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.347.990 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.011 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.034 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.083 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.108 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.126 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.143 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.173 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.193 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.212 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.232 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.251 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.273 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.375 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.480 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.608 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.629 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.736 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.348.942 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.045 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.146 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.172 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.210 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.235 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.453 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.553 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.658 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.705 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.808 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.349.934 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.140 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.244 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.348 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.371 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.469 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.490 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.598 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.638 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.656 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.674 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.691 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.709 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.726 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.764 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.780 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.807 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.825 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.844 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.860 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.955 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.977 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.350.995 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.093 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.117 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.135 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.153 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.171 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.189 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.225 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.243 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.263 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.395 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.468 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.670 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.351.814 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.132 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.221 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.315 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.480 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.549 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.624 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.732 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.839 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.352.937 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.353.016 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.353.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.353.158 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.353.177 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.353.443 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.362.195 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.366.641 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.387.926 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.389.377 [plugin_manager.cc:554]619435 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619435,python3.9):2025-04-03-09:35:44.389.402 [plugin_manager.cc:134]619435 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.389.461 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.389.477 [plugin_manager.cc:145]619435 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.389.489 [plugin_manager.cc:557]619435 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.389.501 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.389.528 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.080 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.145 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.179 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.269 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.286 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.345 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.360 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.374 [opp_so_manager.cc:66]619435 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.386 [plugin_manager.cc:616]619435 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.398 [plugin_manager.cc:108]619435 GetOppPath:Enter get opp path schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.417 [plugin_manager.cc:117]619435 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.433 [plugin_manager.cc:624]619435 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.449 [plugin_manager.cc:206]619435 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.796 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.390.834 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.390.849 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.390.861 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.390.902 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.223 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.257 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.281 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.294 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.330 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.368 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.382 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.395 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.408 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.424 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.738 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.771 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.784 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.391.796 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.391.829 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.141 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.183 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.198 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.210 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.246 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.290 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.303 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.315 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.326 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.338 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.650 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.682 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.697 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.392.710 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.392.742 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.057 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.088 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.102 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.113 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.145 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.180 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.192 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.204 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.225 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.238 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.548 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.579 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.595 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.606 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.636 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.393.948 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.980 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.393.993 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.005 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.035 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.070 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.083 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.095 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.107 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.120 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.426 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.472 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.485 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.500 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.532 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.844 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.876 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.888 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.900 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.394.933 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.966 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.980 [plugin_manager.cc:1026]619435 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.394.992 [plugin_manager.cc:322]619435 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.004 [plugin_manager.cc:212]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.020 [plugin_manager.cc:215]619435 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.030 [plugin_manager.cc:218]619435 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.058 [plugin_manager.cc:422]619435 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.072 [plugin_manager.cc:155]619435 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.092 [plugin_manager.cc:158]619435 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.104 [plugin_manager.cc:426]619435 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.408 [plugin_manager.cc:250]619435 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.395.442 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.395.456 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.395.468 [plugin_manager.cc:1063]619435 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.501 [plugin_manager.cc:439]619435 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.797 [plugin_manager.cc:996]619435 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.824 [plugin_manager.cc:1011]619435 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.842 [plugin_manager.cc:1015]619435 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.395.892 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.395.992 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.396.007 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.396.017 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.397.696 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff240762c0, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.397.735 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.397.756 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.397.771 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.397.787 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.077 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff2407a978, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.101 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.117 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.134 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.148 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.414 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.398.443 [op_impl_registry_holder_manager.cc:216]619435 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.458 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.496 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.748 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.398.802 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.913 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.938 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.398.950 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.326 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.363 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.382 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.398 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.413 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.684 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff240279b0, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.708 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.724 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.739 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.400.754 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.026 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff2402ada0, registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.047 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.062 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.077 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.095 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.337 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.362 [op_impl_registry_holder_manager.cc:216]619435 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.387 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.420 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.750 [op_impl_space_registry.cc:84]619435 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.767 [op_impl_space_registry.cc:85]619435 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.779 [op_impl_space_registry.cc:86]619435 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.790 [op_impl_space_registry.cc:87]619435 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.801 [op_impl_space_registry.cc:93]619435 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.819 [op_impl_space_registry.cc:84]619435 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.835 [op_impl_space_registry.cc:85]619435 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.846 [op_impl_space_registry.cc:86]619435 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.855 [op_impl_space_registry.cc:87]619435 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.866 [op_impl_space_registry.cc:93]619435 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619435,python3.9):2025-04-03-09:35:44.401.876 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.923 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.961 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.401.994 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.090 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.104 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.116 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.686 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.402.707 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.402.749 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.842 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.855 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.402.867 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.403.521 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.403.543 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.403.583 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.403.619 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.403.652 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.403.741 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.403.755 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.403.767 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.404.224 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.404.256 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.404.295 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.404.385 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.404.399 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.404.410 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.404.975 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.404.996 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.405.035 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.405.069 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.405.102 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.190 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.205 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.216 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.643 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.405.662 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.405.714 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.800 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.814 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.405.825 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.406.383 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.406.403 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.406.444 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.406.479 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.406.511 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.406.598 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.406.610 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.406.621 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.407.038 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.057 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.095 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.407.195 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.407.208 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.407.219 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.407.769 [op_impl_space_registry.cc:240]619435 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.794 [opp_so_manager.cc:48]619435 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.833 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.865 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.897 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.921 [opp_so_manager.cc:126]619435 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.407.949 [opp_so_manager.cc:123]619435 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.408.027 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.408.040 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.408.083 [plugin_manager.cc:1126]619435 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.408.096 [plugin_manager.cc:1130]619435 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.408.107 [opp_so_manager.cc:45]619435 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.421.571 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.422.346 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.422.416 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.422.446 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.425.197 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.452.284 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.452.282 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.452.321 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.452.340 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.348 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.364 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.393 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.409 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.452.439 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.452.440 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.452.465 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.452.473 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.488 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.452.508 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.544.197 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.544.744 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.544.827 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.544.897 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.544.963 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.030 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.105 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.137 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.221 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.242 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.260 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.276 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.457 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.528 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.594 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.740 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.762 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.828 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.849 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.866 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.545.937 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.013 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.078 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.145 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.215 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.291 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.356 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.429 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.494 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.561 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.646 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.723 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.788 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.854 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.921 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.546.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.547.052 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.552.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.552.802 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.552.918 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.208 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.338 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.460 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.580 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.737 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.553.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.018 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.183 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.293 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.405 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.528 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.775 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.802 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.554.972 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.244 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.374 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.507 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.727 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.555.894 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.021 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.130 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.262 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.389 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.509 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.632 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.744 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.861 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.556.975 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.004 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.238 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.359 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.509 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.744 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffef7fc6360, registered count:5 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.771 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.789 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.806 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.834 [op_check.cc:71]619435 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.557.981 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.124 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.343 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.498 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.611 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.637 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.764 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.790 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.558.907 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.016 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.039 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.144 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.260 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.494 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.832 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.559.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.170 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.284 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.392 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.512 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.639 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.560.849 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.003 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.161 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.328 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.449 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.569 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.795 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.561.912 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.184 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.354 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.381 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffef8050e08, registered count:7 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.587 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffef8059298, registered count:7 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.719 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.562.987 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.101 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.210 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.343 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.462 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.573 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.687 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.796 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.830 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.563.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.074 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.311 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.422 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.541 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.773 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.921 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.946 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.972 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.564.993 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.151 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.271 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.394 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.511 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.620 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.728 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.779 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.565.896 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.127 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.330 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.447 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.576 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.797 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.566.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.041 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.065 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.183 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.291 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.607 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.724 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.567.958 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.068 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.200 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.310 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.424 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.629 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.746 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.568.881 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.373 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.411 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.434 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.568 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.681 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.793 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.571.925 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.049 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.170 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.290 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.399 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.548 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.686 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.572.959 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.067 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.183 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.294 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.543 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.835 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.573.993 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.574.112 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.574.242 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.574.366 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.575.062 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.575.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.576.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.576.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.577.399 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.577.530 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.577.658 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.577.806 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.577.929 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.578.239 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.580.540 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.581.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.581.588 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.581.745 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.582.782 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.582.945 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.583.110 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.583.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.583.720 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.583.836 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.583.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.057 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.202 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.331 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.440 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.552 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.667 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.791 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.584.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.074 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.192 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.387 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.504 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.614 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.725 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.585.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.073 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.192 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.303 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.326 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.515 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.630 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.749 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.586.934 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.053 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.215 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.329 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.456 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.594 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.713 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.734 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.842 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.949 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.587.972 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.098 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.228 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.350 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.531 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.660 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.588.855 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.589.008 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.589.415 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.589.608 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.589.791 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.589.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.092 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.212 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.329 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.437 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.462 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.756 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.870 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.590.988 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.103 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.220 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.329 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.350 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.460 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.591.692 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.592.763 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.595.226 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.595.449 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.595.653 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.595.996 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.596.180 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.596.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.181 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.348 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.478 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.631 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.796 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.597.931 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.080 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.247 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.390 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.534 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.646 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.598.747 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.057 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.084 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.105 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.124 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.145 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.170 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.205 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.227 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.245 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.284 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.304 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.323 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.340 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.358 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.379 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.398 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.416 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.436 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.455 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.472 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.489 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.507 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.524 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.541 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.560 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.601 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.618 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.636 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.654 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.695 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.714 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.733 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.767 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.787 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.804 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.821 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.839 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.856 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.874 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.894 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.911 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.932 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.969 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.599.986 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.004 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.022 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.040 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.057 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.075 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.092 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.109 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.285 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.445 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.619 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.644 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.664 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.682 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.699 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.838 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.600.952 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.064 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.169 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.294 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.401 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.562 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.679 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.837 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.601.960 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.081 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.104 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.128 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.233 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.254 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.273 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.385 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.543 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.706 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.602.887 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.004 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.191 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.305 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.509 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.622 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.733 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.754 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.773 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.603.919 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.093 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.264 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.411 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.601 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.604.870 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.023 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.184 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.362 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.500 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.612 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.722 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.840 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.605.961 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.122 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.285 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.396 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.502 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.616 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.606.862 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.607.315 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.607.782 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.607.907 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.021 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.043 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.174 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.295 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.407 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.517 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.636 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.868 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.608.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.139 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.249 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.365 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.542 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.653 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.781 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.609.901 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.094 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.273 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.384 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.493 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.610 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.726 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.846 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.610.953 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.063 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.218 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.327 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.493 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.752 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.611.951 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.123 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.312 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.547 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.750 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.975 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.612.997 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.031 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.054 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.074 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.101 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.120 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.613.910 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.614.326 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.614.748 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.614.936 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.615.145 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.615.555 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.615.676 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.615.856 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.618.653 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.620.520 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.620.653 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.620.767 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.620.886 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.166 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.329 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.519 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.863 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.892 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.913 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.932 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.621.950 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.131 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.157 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.330 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.499 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.669 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.622.836 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.032 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.057 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.219 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.386 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.564 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.743 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.623.916 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.070 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.098 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.269 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.455 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.479 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.590 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.710 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.739 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.859 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.624.884 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.004 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.121 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.245 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.376 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.481 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.583 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.778 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.625.978 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.626.368 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.626.757 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.626.958 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.627.302 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.627.500 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.627.693 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.627.811 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.627.938 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.628.106 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.628.133 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.628.180 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.628.205 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.629.323 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.629.779 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.629.946 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.113 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.267 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.293 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.440 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.595 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.746 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.784 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.630.941 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.631.101 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.631.247 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.631.406 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.631.558 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.631.974 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.632.342 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.632.587 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.632.778 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.633.864 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.634.289 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.634.846 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.012 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.164 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.282 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.447 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.604 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.628 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.733 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.635.756 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.088 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.213 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.362 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.518 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.636.964 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.146 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.307 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.421 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.531 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.696 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.637.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.065 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.090 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.247 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.270 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.434 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.638.615 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.640.089 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.645.779 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.645.822 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.648.589 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.648.853 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.649.145 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.649.467 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.649.675 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.649.882 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.048 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.213 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.423 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.565 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.651 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.650.903 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.651.030 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.651.192 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.651.677 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.651.829 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.651.989 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.015 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.125 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.255 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.447 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.629 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.813 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.652.974 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.085 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.200 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.222 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.419 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.606 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.760 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.653.873 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.032 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.055 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.182 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.311 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.334 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.353 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.500 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.692 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.654.782 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.029 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.212 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.239 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.263 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.441 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.655.466 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.320 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.405 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.432 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.508 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.576 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.595 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.667 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.831 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.850 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.957 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.976 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.658.991 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.659.007 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.659.187 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.659.205 [tuning_tiling_registry.cc:30]619435 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.660.423 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffef89950b0, registered count:7 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.660.454 [op_check.cc:24]619435 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffef8995310, registered count:5 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.660.848 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.661.356 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.663.070 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.663.239 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.663.316 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.663.699 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.664.151 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.693.794 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.693.927 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.694.110 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.694.194 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.694.351 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.694.485 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.112 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.201 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.475 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.561 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.725 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.695.959 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.696.133 [op_impl_registry.cc:122]619435 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619435,python3.9):2025-04-03-09:35:44.696.875 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.702.599 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619435,python3.9):2025-04-03-09:35:44.704.577 [op_impl_registry_holder_manager.cc:208]619435 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.777.673 [op_impl_space_registry.cc:109]619435 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.778.981 [op_impl_space_registry.cc:84]619435 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.779.005 [op_impl_space_registry.cc:85]619435 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.779.015 [op_impl_space_registry.cc:86]619435 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.779.027 [op_impl_space_registry.cc:87]619435 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.779.038 [op_impl_space_registry.cc:93]619435 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.075 [op_impl_space_registry.cc:273]619435 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.554 [plugin_manager.cc:564]619435 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.578 [plugin_manager.cc:134]619435 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.780.632 [file_utils.cc:47]619435 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.780.646 [plugin_manager.cc:145]619435 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619435,python3.9):2025-04-03-09:35:44.780.658 [plugin_manager.cc:567]619435 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.721 [mem_manager.cc:31]619435 Initialize:MemManager Initialize. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.742 [rts_caching_mem_allocator.cc:131]619435 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.755 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.767 [graph_mem_allocator.cc:17]619435 Initialize:MemoryAllocator::Initialize. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.779 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.790 [graph_mem_allocator.cc:17]619435 Initialize:MemoryAllocator::Initialize. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.801 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.812 [caching_allocator.cc:391]619435 FreeBlocks:Free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.824 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.838 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.850 [caching_allocator.cc:391]619435 FreeBlocks:Free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.859 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.885 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.898 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.910 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.920 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.930 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.940 [session_scope_mem_allocator.cc:73]619435 FreeAllMemory:Free all memory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.952 [mem_manager.cc:102]619435 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.780.963 [session_scope_mem_allocator.cc:73]619435 FreeAllMemory:Free all memory [INFO] GE(619435,python3.9):2025-04-03-09:35:44.781.006 [dump_manager.cc:152]619435 Init:exception dump is not enabled [INFO] GE(619435,python3.9):2025-04-03-09:35:44.781.044 [ge_executor.cc:315]619435 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.061 [acl.cpp:321]619435 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.073 [dump.cpp:841]619435 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.222 [dump.cpp:759]619435 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.235 [dump.cpp:594]619435 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.246 [dump.cpp:224]619435 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.280 [dump.cpp:703]619435 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.294 [dump.cpp:771]619435 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.781.306 [dump.cpp:703]619435 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.363 [adx_datadump_server.cpp:77][tid:619435] start to do dump init [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.380 [adx_dump_record.cpp:210][tid:619435] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.393 [adx_dump_record.cpp:214][tid:619435] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.610 [adx_server_manager.cpp:149][tid:619712] server manager add component (10) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.628 [adx_dump_record.cpp:627][tid:619713] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.640 [adx_server_manager.cpp:165][tid:619712] server manager components init successfully [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.791 [adx_server_manager.cpp:203][tid:619714] Run Server(0) Process [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.861 [adx_server_manager.cpp:369][tid:619714] device up 0 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.781.880 [hdc_comm_opt.cpp:59][tid:619714] open device[0] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.063 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.089 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.104 [hdc_api.cpp:119][tid:619714] logDevId 0 create HDC server successfully [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.146 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.160 [adx_server_manager.cpp:369][tid:619714] device up 1 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.172 [hdc_comm_opt.cpp:59][tid:619714] open device[1] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.319 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.334 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.344 [hdc_api.cpp:119][tid:619714] logDevId 1 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.355 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.368 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.379 [adx_server_manager.cpp:369][tid:619714] device up 2 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.389 [hdc_comm_opt.cpp:59][tid:619714] open device[2] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.522 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.537 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.549 [hdc_api.cpp:119][tid:619714] logDevId 2 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.560 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.572 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.585 [adx_server_manager.cpp:369][tid:619714] device up 3 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.594 [hdc_comm_opt.cpp:59][tid:619714] open device[3] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.739 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.753 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.765 [hdc_api.cpp:119][tid:619714] logDevId 3 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.776 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.788 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.798 [adx_server_manager.cpp:369][tid:619714] device up 4 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.809 [hdc_comm_opt.cpp:59][tid:619714] open device[4] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.943 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.782.957 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.975 [hdc_api.cpp:119][tid:619714] logDevId 4 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.986 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.782.998 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.012 [adx_server_manager.cpp:369][tid:619714] device up 5 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.021 [hdc_comm_opt.cpp:59][tid:619714] open device[5] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.094 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.106 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.118 [hdc_api.cpp:119][tid:619714] logDevId 5 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.129 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.142 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.152 [adx_server_manager.cpp:369][tid:619714] device up 6 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.162 [hdc_comm_opt.cpp:59][tid:619714] open device[6] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.296 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.309 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.321 [hdc_api.cpp:119][tid:619714] logDevId 6 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.329 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.341 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.351 [adx_server_manager.cpp:369][tid:619714] device up 7 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.363 [hdc_comm_opt.cpp:59][tid:619714] open device[7] server[13] [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.437 [drv_log_user.c:623][ascend][curpid:619435,619714][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.783.451 [hdc_server.c:363][ascend][curpid:619435,619714][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.463 [hdc_api.cpp:119][tid:619714] logDevId 7 create HDC server successfully [WARNING] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.474 [adx_hdc_epoll.cpp:34][tid:619714] hdc epoll has been created. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.783.487 [adx_server_manager.cpp:111][tid:619714] create server info [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.784.723 [adx_server_manager.cpp:457][tid:619435] The server is initialized after waiting 3 times. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.784.743 [ge_executor.cc:1349]619435 SetDump:Start to set dump config [INFO] GE(619435,python3.9):2025-04-03-09:35:44.784.757 [dump_manager.cc:201]619435 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.784.787 [dump_manager.cc:261]619435 SetExceptionDump:Enable brief exception dump. [INFO] GE(619435,python3.9):2025-04-03-09:35:44.784.816 [ge_executor.cc:1355]619435 SetDump:Set dump config successfully [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.784.844 [dump_manager.cpp:146][tid:619435] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:44.786.247 [dump_manager.cpp:301][tid:619435] Register exception callback, registered: 1 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.266 [dump.cpp:825]619435 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.284 [acl.cpp:327]619435 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.295 [acl.cpp:329]619435 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.318 [acl_resource_manager.cpp:248]619435 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.412 [acl_resource_manager.cpp:275]619435 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.427 [acl_resource_manager.cpp:278]619435 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.437 [acl.cpp:335]619435 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.449 [acl.cpp:336]619435 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.509 [json_parser.cpp:260]619435 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.523 [acl.cpp:343]619435 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:44.786.545 [prof_atls_plugin.cpp:210] (tid:619435) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.558 [acl.cpp:351]619435 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.569 [profiling.cpp:141]619435 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.581 [profiling.cpp:135]619435 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.642 [profiling.cpp:167]619435 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619435.json], errorCode = 0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.656 [profiling.cpp:172]619435 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619435.json] is empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.667 [profiling.cpp:176]619435 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.677 [profiling.cpp:179]619435 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.786.689 [profiling.cpp:92]619435 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:44.787.037 [ai_drv_dev_api.cpp:316] >>> (tid:619435) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:44.787.146 [msprof_callback_impl.cpp:132] >>> (tid:619435) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.161 [profiling.cpp:122]619435 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.172 [profiling.cpp:186]619435 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.181 [hash_utils.cpp:108]619435 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619435.json [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.228 [hash_utils.cpp:128]619435 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619435.json, hash: 3917491750093904922 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.243 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.261 [acl.cpp:280]619435 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.272 [acl.cpp:385]619435 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.787.288 [runtime.cc:5394] 619435 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.787.302 [api_impl.cc:7407] 619435 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.787.331 [api_impl.cc:4211] 619435 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.787.346 [acl.cpp:420]619435 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.787.517 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.787.597 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.787.654 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.787.672 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.792194] [info] [619435] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist mki_log delete old file:/home/jenkins/atb/log/atb_1952700_20250328090704.log mki_log delete old file:/home/jenkins/atb/log/atb_1952735_20250328090704.log [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:44.792.875 [prof_atls_plugin.cpp:210] (tid:619435) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.793.612 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.793.638 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.793.705 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 4, rank size = 8. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.793.721 [device.cpp:143]619435 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.742 [api_impl.cc:2999] 619435 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.756 [device.cc:306] 619435 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.771 [runtime.cc:3393] 619435 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.786 [runtime.cc:3560] 619435 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.808 [api_impl.cc:2987] 619435 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.822 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.839 [api_impl.cc:2992] 619435 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.856 [api_impl.cc:3025] 619435 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.793.868 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.793.880 [acl.cpp:121]619435 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.793.901 [device.cpp:154]619435 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.793.947 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.793.963 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is _npu_log, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.981 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.793.993 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.794.005 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.794.044 [queue_client.c:151][ascend][curpid:619435,619435][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619435) [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.794.216 [queue_client.c:305][ascend][curpid:619435,619435][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=4; vfid=0; hostpid=619435; master_pid=619435; proc_type=0; devpid=30682). [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.798.174 [queue_client.c:362][ascend][curpid:619435,619435][drv][queuemng][QueueInit]QueueInit device. (devid=4; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.368 [npu_driver.cc:4114] 619435 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.386 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.400 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=_npu_log. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.798.690 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.798.707 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.798.842 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.798.869 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.798.892 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.911 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.798.912 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.933 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.939 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.949 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.798.960 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.002 [npu_driver.cc:4087] 619435 MemQueueInit: repeated init [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.015 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.026 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=ms_tensor_dump. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.799.327 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.344 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.452 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.799.473 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.486 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.493 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.506 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.509 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.520 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.523 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.535 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.568 [npu_driver.cc:4087] 619435 MemQueueInit: repeated init [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.582 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.597 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=ms_tensor_summary. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.799.779 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.794 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.895 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.799.922 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.932 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.799.939 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.952 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.956 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.966 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.968 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.799.981 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.012 [npu_driver.cc:4087] 619435 MemQueueInit: repeated init [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.024 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.034 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=ms_image_summary. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.800.191 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.208 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.386 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.800.412 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.412 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.430 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.436 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.447 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.449 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.460 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.469 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.501 [npu_driver.cc:4087] 619435 MemQueueInit: repeated init [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.513 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.524 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=ms_scalar_summary. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.800.685 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.700 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.800 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.800.823 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.826 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.800.840 [tensor_data_transfer.cpp:898]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.847 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.856 [api_impl.cc:6077] 619435 MemQueueInit: Start to init queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.858 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.866 [npu_driver.cc:4076] 619435 MemQueueInit: init mem queue, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.876 [npu_driver.cc:4026] 619435 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.907 [npu_driver.cc:4087] 619435 MemQueueInit: repeated init [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.919 [api_impl.cc:6040] 619435 MemQueueCreate: Start to create queue on devId 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.800.930 [npu_driver.cc:3931] 619435 MemQueueCreate: create mem queue, deviceId=4, name=ms_histogram_summary. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.801.103 [queue_interface.c:182][ascend][curpid:619435,619435][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.801.117 [tensor_data_transfer.cpp:940]619435 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 4, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.801.211 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.801.235 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.801.248 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.801.253 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.801.266 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.801.393 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.801.424 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.449 [plugin_manager.cc:25] [619435]hcom ops plugin init start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.480 [plugin_manager.cc:38] [619435]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.498 [plugin_manager.cc:42] [619435]hcom running normal mode. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.541 [hcom_plugin.cc:50] [619435]Initialize start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.552 [hcom_plugin.cc:52] [619435]initializeCount_ : 1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.599 [hcom_plugin.cc:60] [619435]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.627 [hcom_plugin.cc:63] [619435]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.640 [hcom_plugin.cc:329] [619435]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.655 [hcom_plugin.cc:281] [619435]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:44.801.676 [hcom_plugin.cc:217] [619435]option profiling mode is not found. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.692 [hcom_plugin.cc:164] [619435][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.706 [hcom_plugin.cc:180] [619435][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.715 [hcom_plugin.cc:244] [619435]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.851 [hcom_plugin.cc:82] [619435]hccl ops plugin init success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.865 [plugin_manager.cc:48] [619435]hcom ops plugin init end. ret[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.877 [hcom_plugin.cc:126] [619435]get hccl kernel info store start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.902 [hcom_plugin.cc:132] [619435]get hccl kernel info store finished. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.801.927 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.801.946 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.801.964 [hcom_executor.cc:29] [619435]Hcom Excutor Initialize start. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.010 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.026 [adapter_rts.cc:234] [619435][hrtGetDevice]deviceLogicId[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.056 [api_c_soc.cc:70] 619435 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.097 [hcom_common.cc:85] [619435][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.120 [hcom_common.cc:88] [619435][HcomGetCurHcomCtx] is no Used deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.143 [hcom.cc:486] [619435]Get Hcom Init Status: [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.167 [hcom_common.cc:85] [619435][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.802.179 [hcom_rpc.cc:1019] [619435]Get Hcom Init Status: [0] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:44.802.199 [hcom_base_funs.cc:36] [619435][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:44.802.225 [hcom_executor.cc:121] [619435][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.300 [api_impl.cc:1120] 619435 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.322 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.394 [stream.cc:2246] 619435 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.446 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.461 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.889 [npu_driver.cc:6125] 619435 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.941 [npu_driver.cc:6137] 619435 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=322, streamId=4, logicCq=322, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.966 [npu_driver.cc:6189] 619435 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=4, logicCq=322. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.983 [npu_driver.cc:5916] 619435 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0071000, addr=dfffe0071000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.802.997 [stream.cc:653] 619435 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.803.057 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.803.076 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.803.387 [stream.cc:291] 619435 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.803.988 [stream.cc:688] 619435 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=4, streamResId=65535 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.804.006 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.804.018 [acl_resource_manager.cpp:816]619435 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.031 [api_impl.cc:1162] 619435 StreamCreate: Succ, flags=1536, stream_id=4, context=aaab02fde2d0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.804.056 [stream.cc:73] [619435]Construct stream by stream type success, ptr[0xaaab0e305f20] ctx[0xaaab02fde2d0], stream id[4], cqId[4], logicCqid[322] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.095 [api_impl.cc:1120] 619435 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.116 [npu_driver.cc:404] 619435 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.144 [stream.cc:2246] 619435 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.182 [stream_sqcq_manage.cc:147] 619435 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.197 [npu_driver.cc:882] 619435 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.432 [npu_driver.cc:6125] 619435 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.479 [npu_driver.cc:6137] 619435 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=323, streamId=5, logicCq=323, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.508 [npu_driver.cc:6189] 619435 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=5, logicCq=323. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.522 [npu_driver.cc:5916] 619435 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0050000, addr=dfffe0050000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.534 [stream.cc:653] 619435 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619435,python3.9):2025-04-03-09:35:44.804.578 [devmm_svm.c:1972][ascend][curpid:619435,619435][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.593 [npu_driver.cc:6080] 619435 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.804.876 [stream.cc:291] 619435 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.513 [stream.cc:688] 619435 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=4, streamResId=65535 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.805.532 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.805.544 [acl_resource_manager.cpp:816]619435 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.556 [api_impl.cc:1162] 619435 StreamCreate: Succ, flags=1536, stream_id=5, context=aaab02fde2d0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.805.574 [stream.cc:73] [619435]Construct stream by stream type success, ptr[0xaaab0e381f80] ctx[0xaaab02fde2d0], stream id[5], cqId[5], logicCqid[323] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:44.805.716 [sal.cc:619] [619727]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.805.748 [hcom_executor.cc:587] [619727]MessageProcessThreadLoop start queType:[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.823 [api_impl.cc:2999] 619727 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.838 [device.cc:306] 619727 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.850 [runtime.cc:3393] 619727 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:44.805.847 [sal.cc:619] [619728]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.864 [runtime.cc:3560] 619727 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.805.875 [hcom_executor.cc:587] [619728]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.877 [api_impl.cc:2987] 619727 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.893 [runtime.cc:5239] 619727 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.901 [api_impl.cc:2999] 619728 SetDevice: device_id=4, deviceMode=0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:44.805.900 [hcom_executor.cc:34] [619435][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.900 [api_impl.cc:2999] 619729 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.911 [api_impl.cc:2992] 619727 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.917 [device.cc:306] 619728 Init: device mode has been already set, mode=0. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:44.805.943 [awatchdog_core.c:175](tid:619727) create watchdog for id : 65543, tid : 619727, timeout : 300s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.940 [api_impl.cc:2999] 619730 SetDevice: device_id=4, deviceMode=0. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.805.944 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.945 [runtime.cc:3393] 619728 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.947 [device.cc:306] 619729 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.963 [api_impl.cc:3025] 619727 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.805.982 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.981 [runtime.cc:3560] 619728 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.988 [runtime.cc:3393] 619729 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.990 [device.cc:306] 619730 Init: device mode has been already set, mode=0. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.805.996 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.805.995 [api_impl.cc:2987] 619728 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.001 [runtime.cc:3560] 619729 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.005 [runtime.cc:3393] 619730 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.012 [runtime.cc:5239] 619728 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.015 [api_impl.cc:2987] 619729 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.021 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.020 [runtime.cc:3560] 619730 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.027 [runtime.cc:5239] 619729 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.029 [api_impl.cc:2992] 619728 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.034 [api_impl.cc:2987] 619730 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.038 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.043 [api_impl.cc:2992] 619729 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.046 [runtime.cc:5239] 619730 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:44.806.048 [awatchdog_core.c:175](tid:619728) create watchdog for id : 65543, tid : 619728, timeout : 300s [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:44.806.059 [awatchdog_core.c:175](tid:619729) create watchdog for id : 65543, tid : 619729, timeout : 300s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.061 [api_impl.cc:3025] 619728 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.063 [api_impl.cc:2992] 619730 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.071 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1153.56 msec. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.073 [api_impl.cc:3025] 619729 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:44.806.096 [awatchdog_core.c:175](tid:619730) create watchdog for id : 65543, tid : 619730, timeout : 300s [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.110 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.112 [api_impl.cc:3025] 619730 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [WARNING] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.127 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.169 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.012 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.209 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.245 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.266 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.076 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.363 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:44.806.385 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:44.806.522 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:44.806.558 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.806.601 [device.cpp:143]619731 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.625 [api_impl.cc:2999] 619731 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.637 [device.cc:306] 619731 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.649 [runtime.cc:3393] 619731 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.661 [runtime.cc:3560] 619731 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.678 [api_impl.cc:2987] 619731 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.690 [runtime.cc:5239] 619731 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.703 [api_impl.cc:2992] 619731 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:44.806.717 [awatchdog_core.c:175](tid:619731) create watchdog for id : 65543, tid : 619731, timeout : 300s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.806.728 [api_impl.cc:3025] 619731 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.806.740 [acl.cpp:115]619731 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.806.751 [acl.cpp:121]619731 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.806.763 [device.cpp:154]619731 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:44.806.796 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.186 msec. [WARNING] DEVICE(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:44.807.092 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 2 [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.960.193 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.960.234 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.960.256 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.960.277 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.960.305 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.960.331 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.960.356 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.964.188 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:44.964.215 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.964.233 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:44.964.247 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.964.263 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:44.964.276 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:44.964.292 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.100.659 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.100.700 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.100.715 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.100.727 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.100.746 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.100.822 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:45.307.343 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:45.307.381 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.557 msec. [WARNING] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:45.307.396 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:45.307.415 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.307.553 [device.cpp:143]619762 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.588 [api_impl.cc:2999] 619762 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.603 [device.cc:306] 619762 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.616 [runtime.cc:3393] 619762 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.631 [runtime.cc:3560] 619762 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.643 [api_impl.cc:2987] 619762 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.654 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.672 [api_impl.cc:2992] 619762 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:45.307.692 [awatchdog_core.c:175](tid:619762) create watchdog for id : 65543, tid : 619762, timeout : 300s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.307.705 [api_impl.cc:3025] 619762 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.307.718 [acl.cpp:115]619762 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.307.730 [acl.cpp:121]619762 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.307.743 [device.cpp:154]619762 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.308.317 [device.cpp:231]619762 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.346 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.545 [api_c_soc.cc:70] 619762 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.308.599 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.617 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.631 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.645 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.660 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.676 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=0. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.308.699 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=0.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.308.734 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.747 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.761 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.773 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.785 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.308.795 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=1. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.308.961 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=1.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.308.996 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.013 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.026 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.038 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.049 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.061 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=2. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.309.350 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=2.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.309.382 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.397 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.414 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.426 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.437 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.309.460 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=3. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.310.085 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=3.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.310.119 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.135 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.146 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.161 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.174 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.192 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=5. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.310.625 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=5.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.310.658 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.672 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.686 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.699 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.711 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.310.723 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=6. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.311.789 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=6.) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.311.822 [memory.cpp:559]619762 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.311.836 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.311.848 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.311.860 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.311.873 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.311.884 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=7. [INFO] DRV(619435,python3.9):2025-04-03-09:35:45.312.832 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=7.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.312.868 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.312.929 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.312.957 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.312.991 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.045 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.064 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.087 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.105 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.122 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.148 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.167 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.182 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.205 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.222 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.238 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.253 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.270 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.293 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.317 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.335 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.350 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.365 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.382 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.398 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.420 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.443 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.471 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.486 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.501 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.514 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.532 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.556 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.573 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.586 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.609 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.629 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.645 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.659 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.685 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.702 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.716 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.749 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.766 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.780 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.794 [runtime.cc:1664] 619762 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.805 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.828 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.845 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.862 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.892 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.916 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.928 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.940 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.953 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.972 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.313.995 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.007 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.019 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.047 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.062 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.088 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.101 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.113 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.129 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.144 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.172 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.186 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.198 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.213 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.228 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.254 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.268 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.283 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.299 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.313 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.339 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.353 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.368 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.381 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.397 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.422 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.432 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.444 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.710 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.759 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.799 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.811 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.824 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.840 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.861 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.901 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.915 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.927 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.941 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.962 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.314.996 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.010 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.023 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.037 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.061 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.098 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.111 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.124 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.136 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.153 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.191 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.203 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.216 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.230 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.253 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.290 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.303 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.323 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.337 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.350 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.389 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.402 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.413 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.427 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.449 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.484 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.495 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.506 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.520 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.537 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.575 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.587 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.598 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.612 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.636 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.675 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.691 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.702 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.718 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.734 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.772 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.786 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.799 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.820 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.840 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.877 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.889 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.902 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.916 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.931 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.968 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.980 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.315.993 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.008 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.028 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.066 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.078 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.092 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.104 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.119 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.171 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.191 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.203 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.224 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.247 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.287 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.302 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.315 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.330 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.357 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.399 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.410 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.423 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.437 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.453 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.489 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.502 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.514 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.529 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.546 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.585 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.600 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.616 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.631 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.648 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.689 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.703 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.716 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.732 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.753 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.789 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.802 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.815 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.829 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.844 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.883 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.910 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.923 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.316.975 [api_impl.cc:1990] 619762 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.317.283 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.317.319 [api_impl.cc:2664] 619762 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.352 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619419, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.377 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619423, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.477 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619427, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.493 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619431, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.508 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619439, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.522 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619443, num=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.537 [npu_driver.cc:3571] 619762 SetIpcMemPid: set ipc mem pid success, name=000973ab00000000000003170400ffd00004, pid[0]=619447, num=1 [2025-04-03 09:35:44.792897] [debug] [619435] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.792908] [debug] [619435] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.307861] [warn] [619762] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.307888] [debug] [619762] [lcal_sock_exchange.cpp:140] curRank: 4 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.307896] [debug] [619762] [lcal_sock_exchange.cpp:272] Client side 4 begin to connect [2025-04-03 09:35:45.308032] [debug] [619762] [lcal_sock_exchange.cpp:93] rank:4 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308293] [debug] [619762] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 4 rank :4 rankSize :8 [2025-04-03 09:35:45.308502] [debug] [619762] [lcal_comm.cpp:406] rank 4 devId: 4, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308514] [info] [619762] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308519] [info] [619762] [lcal_comm.cpp:242] rank 4/8 running devId:4 [2025-04-03 09:35:45.308561] [debug] [619762] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308590] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308723] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308985] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309374] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310108] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310649] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311812] [debug] [619762] [lcal_comm.cpp:340] 4 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312847] [debug] [619762] [lcal_comm.cpp:372] EnablePeerAccess succeed4 [2025-04-03 09:35:45.316939] [debug] [619762] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 4 [2025-04-03 09:35:45.316962] [debug] [619762] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.317304] [debug] [619762] [lcal_comm.cpp:455] peerMem[rank4], allocate finished. [2025-04-03 09:35:45.340250] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340266] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340273] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340278] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340285] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340294] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340303] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340310] [debug] [619762] [lcal_comm.cpp:472] rank : 4, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340318] [debug] [619762] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340548] [debug] [619762] [lcal_comm.cpp:557] rank 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340801] [debug] [619762] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340812] [debug] [619762] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340820] [debug] [619762] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340826] [debug] [619762] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340848] [debug] [619762] [lcal_comm.cpp:515] rank[INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.340.908 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.356.336 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.356.354 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.357.214 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.357.228 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.357.810 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.357.823 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.358.356 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.358.368 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.359.329 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.359.345 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.111 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.134 [api_impl.cc:4286] 619762 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.641 [npu_driver.cc:3195] 619762 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.685 [api_impl.cc:1990] 619762 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.707 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.360.723 [memory.cpp:421]619762 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:45.360.880 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.360.916 [device.cpp:143]619762 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.935 [api_impl.cc:2999] 619762 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.947 [device.cc:306] 619762 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.960 [runtime.cc:3393] 619762 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.975 [runtime.cc:3560] 619762 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.360.988 [api_impl.cc:2987] 619762 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.361.001 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.361.015 [api_impl.cc:2992] 619762 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.361.029 [api_impl.cc:3025] 619762 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.361.043 [acl.cpp:115]619762 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.361.056 [acl.cpp:121]619762 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.361.072 [device.cpp:154]619762 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [WARNING] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:45.361.103 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.361.144 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.157 [adapter_rts.cc:198] [619762][hrtGetDeviceRefresh]deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.171 [externalinput.cc:2065] [619762]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.190 [externalinput.cc:490] [619762]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.207 [externalinput.cc:429] [619762]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.223 [externalinput.cc:543] [619762]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.238 [externalinput.cc:622] [619762]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.263 [externalinput.cc:714] [619762]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.279 [externalinput.cc:821] [619762]HCCL_IF_IP is not set [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.290 [externalinput.cc:877] [619762]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.302 [externalinput.cc:844] [619762]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.313 [externalinput.cc:728] [619762]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.325 [externalinput.cc:758] [619762]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.336 [externalinput.cc:806] [619762]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.349 [externalinput.cc:1508] [619762]HCCL_ALGO is not set [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.362 [externalinput.cc:1671] [619762][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.374 [externalinput.cc:462] [619762]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.386 [externalinput.cc:1519] [619762]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.400 [externalinput.cc:1554] [619762]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.361.511 [api_c_soc.cc:70] 619762 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.528 [externalinput.cc:1599] [619762]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.548 [externalinput.cc:1633] [619762]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.561 [externalinput.cc:1778] [619762]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.573 [externalinput.cc:1807] [619762]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.587 [externalinput.cc:515] [619762]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.599 [externalinput.cc:1841] [619762]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.612 [externalinput.cc:1929] [619762]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.625 [externalinput.cc:1949] [619762]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.639 [externalinput.cc:1969] [619762]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.662 [externalinput.cc:2005] [619762]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.678 [externalinput.cc:2036] [619762]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.694 [externalinput.cc:2209] [619762]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.709 [externalinput.cc:2178] [619762][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.724 [externalinput.cc:889] [619762]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.739 [externalinput.cc:914] [619762]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.752 [externalinput.cc:2240] [619762]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.780 [externalinput.cc:2308] [619762]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.796 [externalinput.cc:2339] [619762]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.811 [externalinput.cc:2296] [619762]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.828 [externalinput.cc:407] [619762]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:45.361.841 [externalinput.cc:672] [619762][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:45.361.863 [env_config.cc:127] [619762]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:45.361.882 [env_config.cc:131] [619762]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.897 [env_config.cc:302] [619762]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.909 [env_config.cc:302] [619762]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.921 [op_base.cc:1091] [619762]Entry-HcclCommInitRootInfoInner:ranks[8], rank[4], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.946 [op_base.cc:128] [619762][HcclGetDeviceId] deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.959 [op_base.cc:144] [619762][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.361.984 [rank_consistentcy_checker.cc:32] [619762][GetInstance] get deviceLogicId[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.362.016 [api_impl.cc:3100] 619762 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.362.029 [api_impl.cc:3111] 619762 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.362.044 [api_impl.cc:3130] 619762 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.362.248 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.362.266 [network_manager.cc:169] [619762]NetworkManager: devicePhyId[4], devicePhyId_[4] deviceLogicId_[4], hasBackup[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.293 [ra_host.c:1594]tid:619762,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.362.317 [adapter_hccp.cc:1450] [619762]set host socket whitelist status[0] success. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.355 [ra_host.c:205]tid:619762,ra_init(205) : Input parameters: phy_id[4], nic_position:[0] hdc_type:[6] [INFO] ROCE(619435,python3.9):2025-04-03-09:35:45.362.414 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.432 [ra_peer.c:776]tid:619762,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[4] start [INFO] ROCE(619435,python3.9):2025-04-03-09:35:45.362.501 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619435,python3.9):2025-04-03-09:35:45.362.571 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.605 [rs.c:327]tid:619762,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619435,python3.9):2025-04-03-09:35:45.362.768 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619435,python3.9):2025-04-03-09:35:45.362.800 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619435,python3.9):2025-04-03-09:35:45.362.814 [rs_ssl.c:1513]tid:619762,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.826 [rs_ssl.c:1529]tid:619762,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.863 [rs_epoll.c:692]tid:619762,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.961 [rs_epoll.c:487]tid:619769,rs_epoll_handle(487) : thread begin! thread_id:281469372854176, pid:619435, ppid:619411 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.986 [rs_epoll.c:713]tid:619762,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.362.998 [rs_epoll.c:623]tid:619770,rs_connect_handle(623) : thread begin! thread_id:281469364400032, pid:619435, ppid:619411 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.363.001 [rs.c:366]tid:619762,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.163000] ms [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.363.023 [rs_epoll.c:415]tid:619769,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.363.041 [rs_epoll.c:511]tid:619769,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.363.042 [rs_epoll.c:641]tid:619770,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.240 [rs.c:471]tid:619762,rs_init(471) : rs init success, chip_id[4] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.269 [ra_peer.c:813]tid:619762,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[4] succ [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.370.283 [adapter_hccp.cc:583] [619762]init ra success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.370.295 [network_manager.cc:222] [619762]NetworkManager nicDeploy[0] deviceLogicId_[4] devicePhyId_[4] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.325 [ra_host.c:1764]tid:619762,ra_get_ifnum(1764) : Input parameters: phy_id[4], nic_position:[0], is_all:[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.343 [ra_peer.c:859]tid:619762,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[4] start [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.367 [ra_peer.c:867]tid:619762,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[4] succ [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.370.384 [adapter_hccp.cc:1378] [619762][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.400 [ra_host.c:1814]tid:619762,ra_get_ifaddrs(1814) : Input parameters: phy_id[4], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.416 [ra_peer.c:874]tid:619762,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[4] start [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.431 [rs_socket.c:2743]tid:619762,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.446 [rs_socket.c:2743]tid:619762,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.462 [rs_socket.c:2743]tid:619762,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.474 [rs_socket.c:2749]tid:619762,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.486 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.515 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.530 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.544 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.557 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.571 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.584 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.595 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.610 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.622 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.636 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.647 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.658 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.667 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.679 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.691 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.703 [rs_socket.c:2749]tid:619762,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.715 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.727 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.738 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.749 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.760 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.773 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.787 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.799 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.808 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.819 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.832 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.844 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.856 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.872 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.885 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.904 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.916 [rs_socket.c:2749]tid:619762,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.930 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.941 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.951 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.962 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.975 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.370.988 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.000 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.011 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.023 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.035 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.049 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.061 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.071 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.082 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.094 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.107 [rs_socket.c:2751]tid:619762,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.119 [ra_peer.c:882]tid:619762,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[4] succ [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.140 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.157 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.173 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.186 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.202 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.216 [adapter_hccp.cc:1407] [619762][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.236 [sal.cc:365] [619762]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.249 [sal.cc:432] [619762]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.266 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:45.371.306 [ra_host.c:340]tid:619762,ra_socket_init_v1(340) : socket init:mode=0 phy_id=4 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.320 [adapter_hccp.cc:818] [619762]socket init v1 success, socketHandle[0xfffea00faba0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.331 [network_manager.cc:1218] [619762]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.341 [network_manager.cc:1257] [619762]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffea00faba0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.393 [adapter_hal.cc:298] [619762]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.416 [sal.cc:593] [619762]IsHostUseDevNic[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.431 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.445 [network_manager.cc:169] [619762]NetworkManager: devicePhyId[4294967295], devicePhyId_[4] deviceLogicId_[4], hasBackup[0] [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.489 [tsd_client.cpp:189][TsdCapabilityGet][tid:619762] TsdCapabilityGet Begin. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.504 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.518 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.531 [process_mode_manager.cpp:1042][CapabilityGet][tid:619762] [ProcessModeManager] enter into CapabilityGet process deviceId[4] type[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.545 [adapter_tdt.cc:66] [619762]Get TsdCapability success. deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.559 [network_manager.cc:175] [619762][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:45.371.577 [network_manager.cc:99] [619762][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.594 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.606 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.616 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619762] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.627 [process_mode_manager.cpp:386][InitTsdClient][tid:619762] [TsdClient] tsd client has been inited before [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.641 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619762] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619435,python3.9):2025-04-03-09:35:45.371.662 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619762] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.674 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619762] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.688 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619762] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.701 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619762] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.732 [version_verify.cpp:114][SpecialFeatureCheck][tid:619762] VersionVerify: previous type[48], supported [INFO] TDT(619435,python3.9):2025-04-03-09:35:45.371.786 [hdc_client.cpp:154][TsdRecvData][tid:619762] recv timeout is:150000 ms [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.476.182 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.476.183 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.476.207 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.476.223 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.232 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.246 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.255 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.272 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.476.280 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.476.296 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.476.299 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.476.320 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.327 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.476.346 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.604.179 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.604.179 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.604.179 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.290 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.305 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.327 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.328 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.343 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.346 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.358 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.361 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.370 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.374 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.381 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.385 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.394 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.401 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.414 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.604.427 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.440 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.604.452 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.636.165 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.636.171 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.636.172 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.270 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.289 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.297 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.306 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.310 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.319 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.323 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.328 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.348 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.350 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.357 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.360 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.364 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.369 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.375 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.636.380 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.390 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.636.402 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.988.175 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.988.173 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.988.205 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:45.988.234 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.243 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.260 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.268 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.280 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.988.290 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.988.302 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.988.308 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:45.988.324 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.335 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:45.988.347 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.316.639 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:46.316.665 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:46.316.678 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:46.316.688 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.436.173 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.436.174 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.436.175 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.274 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.290 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.302 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.304 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.317 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.319 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.329 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.332 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.339 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.343 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.345 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.351 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.359 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.362 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.371 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.436.382 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.403 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.436.415 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.468.168 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.468.170 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.468.175 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.253 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.265 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.278 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.283 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.294 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.299 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.304 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.310 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.314 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.314 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.323 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.334 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.336 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.349 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.351 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.468.364 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.375 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.468.386 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.500.173 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.500.185 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.500.204 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:46.500.242 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.253 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.263 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.273 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.283 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.500.293 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.500.305 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.500.313 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:46.500.327 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.338 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:46.500.349 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.012.175 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.012.181 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.012.207 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.012.219 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.230 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.239 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.251 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.260 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.012.274 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.012.281 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.012.293 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.012.302 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.318 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.012.325 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.268.176 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.268.176 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.268.180 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.267 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.281 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.291 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.293 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.305 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.307 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.317 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.320 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.328 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.332 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.332 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.338 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.347 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.349 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.358 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.268.369 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.381 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.268.392 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.300.172 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.300.173 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.300.203 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.251 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.264 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.280 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.282 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.292 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.299 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.302 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.313 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.315 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.324 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.336 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.336 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.348 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.363 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.384 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.300.397 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.413 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.300.426 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.524.183 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.524.182 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.524.207 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:47.524.221 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.233 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.245 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.269 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.281 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.524.291 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.524.305 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.524.315 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:47.524.326 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.336 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.524.348 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.731.037 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.731.045 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619762] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.731.071 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619762] [TsdClient] DeviceMsgProc recvMsg realDeviceId[4] msgType[49] localDevId[4] rspCode[0]heterogeneousSubPid[30720], tsdSupportLevel_[63] [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.731.091 [hdc_client.cpp:172][TsdRecvData][tid:619762] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.731.105 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619762] OpenSubProc success stype:0, pid:30720 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.731.119 [adapter_tdt.cc:39] [619762]Open TsdProcess success. deviceLogicId[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.731.119 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.731.129 [network_manager.cc:118] [619762][TsdProcessOpen]hrtTsdProcessOpen success, subPid[30720], devicePhyId_[4], deviceLogicId_[4], hasBackup[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.731.142 [network_manager.cc:180] [619762][NetworkManager][Init]open tsd by process success, devicePhyId[4], deviceLogicId_[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.731.184 [ra_host.c:205]tid:619762,ra_init(205) : Input parameters: phy_id[4], nic_position:[1] hdc_type:[18] [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.731.198 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619435,python3.9):2025-04-03-09:35:47.731.216 [devdrv_hccl.c:103][ascend][curpid:619435,619762][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619435) [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.731.231 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.731.245 [ra_hdc.c:580]tid:619762,ra_hdc_init(580) : hdc init start! logic id is 4, phy id is 4, hdc_type is 18 [INFO] DRV(619435,python3.9):2025-04-03-09:35:47.731.413 [hdc_core.c:1982][ascend][curpid:619435,619762][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4265; pid=619435) [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.731.664 [ra_hdc.c:616]tid:619762,ra_hdc_init(616) : hdc init OK! phy_id[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.735.496 [adapter_hccp.cc:583] [619762]init ra success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.735.512 [network_manager.cc:222] [619762]NetworkManager nicDeploy[1] deviceLogicId_[4] devicePhyId_[4] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.735.535 [topoinfo_detect.cc:534] [619762]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.735.553 [ra_host.c:1764]tid:619762,ra_get_ifnum(1764) : Input parameters: phy_id[4], nic_position:[1], is_all:[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.735.936 [adapter_hccp.cc:1538] [619762][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.735.952 [ra_host.c:1814]tid:619762,ra_get_ifaddrs(1814) : Input parameters: phy_id[4], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.301 [adapter_hccp.cc:1568] [619762][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth4] ip[29.105.24.184%eth4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.319 [adapter_hccp.cc:1568] [619762][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth4] ip[fe80::2287:ecff:fe02:45b0%eth4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.338 [topoinfo_detect.cc:577] [619762]select AF_INET family as device socket family. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.353 [topoinfo_detect.cc:756] [619762]no backup device ip: use 0 as device ip. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.364 [topoinfo_detect.cc:294] [619762][HCCL_TRACE]SetupAgent rankNum[8], rank[4], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[4], phydevId[4], deviceIp[29.105.24.184] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.398 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.414 [hccl_socket.cc:516] [619762][Get][NicHandleInfo]phyId[4], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.428 [hccl_socket.cc:217] [619762][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281469072157600], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.736.442 [adapter_hccp.cc:1050] [619762]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.458 [ra_host.c:759]tid:619762,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[4], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.488 [rs_socket.c:1810]tid:619762,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.506 [rs_socket.c:1819]tid:619762,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619435,python3.9):2025-04-03-09:35:47.736.524 [rs_socket.c:132]tid:619762,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.540 [rs_socket.c:1880]tid:619762,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.796 [rs_socket.c:1577]tid:619770,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:57419, fd:54 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.826 [rs_drv_socket.c:59]tid:619770,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:54 begin [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.925 [rs_drv_socket.c:98]tid:619770,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:57419, server addr 192.168.0.44:60008, fd:54 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.948 [rs_socket.c:1427]tid:619770,rs_conn_cost_time(1427) : socket [54] connect success! cost [0.408000] ms [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.736.989 [rs_socket.c:1392]tid:619770,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.737.633 [topoinfo_exchange_agent.cc:212] [619762]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.737.695 [topoinfo_exchange_agent.cc:236] [619762]local rank[4] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.737.715 [topoinfo_exchange_agent.cc:43] [619762]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.737.731 [topoinfo_exchange_agent.cc:345] [619762]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.737.830 [topoinfo_exchange_agent.cc:77] [619762]topo exchange client send rank basic info success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.801.027 [api_impl.cc:2096] 619762 HostMalloc: size=2485 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.801.076 [api_error.cc:1260] 619762 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:47.801.367 [api_error.cc:1278] 619762 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.387 [topoinfo_exchange_agent.cc:80] [619762]topo exchange client get rank basic info success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.442 [topoinfo_detect.cc:303] [619762]topo detect completed. myrank[4], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.475 [config.cc:198] [619762]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.498 [config.cc:210] [619762]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.514 [config.cc:210] [619762]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.527 [config.cc:210] [619762]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.540 [config.cc:210] [619762]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.553 [config.cc:210] [619762]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.566 [config.cc:210] [619762]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.579 [config.cc:210] [619762]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.594 [config.cc:210] [619762]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.801.620 [adapter_hccp.cc:1067] [619762]ra socket batch close fdhandle[0xfffea0054bf0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.801.635 [ra_host.c:796]tid:619762,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[4], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.801.651 [rs_socket.c:1937]tid:619762,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:54, state:9 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.848.198 [adapter_hccp.cc:1099] [619762]ra socket batch close success,take time [46565]us [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:47.848.218 [hccl_socket.cc:232] [619762][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.848.238 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.256 [ra_host.c:404]tid:619762,ra_socket_deinit(404) : Input parameters: phy_id[4] family[2] local_ip[192.168.0.44] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.274 [rs.c:1629]tid:619762,rs_socket_deinit(1629) : rs socket deinit start, phy_id:4 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.287 [rs.c:1641]tid:619762,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.307 [rs.c:1653]tid:619762,rs_socket_deinit(1653) : socket deinit success, phy_id:4, local_ip:192.168.0.44 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.848.322 [network_manager.cc:1233] [619762]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.848.337 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.353 [ra_host.c:248]tid:619762,ra_deinit(248) : Input parameters: phy_id[4], nic_position:[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.365 [ra_peer.c:821]tid:619762,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[4] start [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.384 [rs.c:1783]tid:619762,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619435,python3.9):2025-04-03-09:35:47.848.390 [rs_epoll.c:523]tid:619769,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:52 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.848.413 [rs_epoll.c:544]tid:619769,rs_epoll_handle(544) : QUIT thread_id:281469372854176, pid:619435, read num:8 [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.947.319 [rs_epoll.c:679]tid:619770,rs_connect_handle(679) : QUIT thread_id:281469364400032, pid:619435 [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.948.686 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.948.711 [rs.c:1810]tid:619762,rs_deinit(1810) : rs_deinit chip_id[4] ok [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.948.724 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.948.738 [ra_peer.c:852]tid:619762,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[4] succ [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.948.751 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.948.781 [network_manager.cc:409] [619762]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.948.798 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.948.814 [ra_host.c:248]tid:619762,ra_deinit(248) : Input parameters: phy_id[4], nic_position:[1] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.948.827 [ra_hdc.c:651]tid:619762,ra_hdc_deinit(651) : hdc deinit start! phy_id[4] [INFO] DRV(619435,python3.9):2025-04-03-09:35:47.948.972 [hdc_client.c:417][ascend][curpid:619435,619762][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4265) [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.949.039 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:47.949.053 [ra_hdc.c:685]tid:619762,ra_hdc_deinit(685) : hdc deinit OK! phy_id[4] [INFO] ROCE(619435,python3.9):2025-04-03-09:35:47.949.068 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.091 [tsd_client.cpp:189][TsdCapabilityGet][tid:619762] TsdCapabilityGet Begin. [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.105 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.118 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.130 [process_mode_manager.cpp:1042][CapabilityGet][tid:619762] [ProcessModeManager] enter into CapabilityGet process deviceId[4] type[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:47.949.143 [adapter_tdt.cc:66] [619762]Get TsdCapability success. deviceLogicId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.154 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.166 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.178 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619762] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.194 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619762] add close subproc:30720, proctype:0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.207 [version_verify.cpp:114][SpecialFeatureCheck][tid:619762] VersionVerify: previous type[60], supported [INFO] TDT(619435,python3.9):2025-04-03-09:35:47.949.235 [hdc_client.cpp:154][TsdRecvData][tid:619762] recv timeout is:150000 ms [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.036.177 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.036.189 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.036.210 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.036.223 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.234 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.245 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.256 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.269 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.036.279 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.036.291 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.036.304 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.036.315 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.327 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.036.352 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.100.173 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.100.180 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.100.181 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.272 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.286 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.298 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.299 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.316 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.315 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.326 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.331 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.337 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.342 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.342 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.354 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.358 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.366 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.370 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.100.382 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.395 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.100.407 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.132.166 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.132.176 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.132.181 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.264 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.281 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.293 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.294 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.303 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.309 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.313 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.322 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.324 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.324 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.335 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.350 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.355 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.369 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.374 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.132.383 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.398 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.132.412 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.201.245 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619762] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.201.266 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619762] [TsdClient] DeviceMsgProc recvMsg realDeviceId[4] msgType[61] localDevId[4] rspCode[0]heterogeneousSubPid[30720], tsdSupportLevel_[63] [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.201.280 [hdc_client.cpp:172][TsdRecvData][tid:619762] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.201.290 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619762] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.201.302 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619762] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.323 [adapter_tdt.cc:52] [619762]Open TsdProcess success. deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.333 [network_manager.cc:405] [619762][DeInit]finish CloseHccpProcess, phy_id[4], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.346 [network_manager.cc:409] [619762]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.356 [topoinfo_detect.cc:555] [619762]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.367 [topoinfo_detect.cc:321] [619762]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.377 [topoinfo_detect.cc:219] [619762]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.401 [rank_consistentcy_checker.cc:32] [619762][GetInstance] get deviceLogicId[4] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:48.201.417 [rank_consistentcy_checker.cc:144] [619762][Record][CannVersion] version information str is empty. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.426 [op_base.cc:2231] [619762]rank table is null, rankTableCrc is 0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.455 [op_base.cc:2219] [619762]params.id.internal [hccl-619435-0-1743644148-hccl_world_group] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.470 [op_base.cc:993] [619762]rootInfo[hccl-619435-0-1743644148-hccl_world_group], params.logiceDevice[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.485 [hccl_comm.cc:98] [619762]hcclComm init workmode [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.201.502 [hccl_comm.cc:954] [619762]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.453 [hccl_comm.cc:1365] [619762]RegistTaskAbortHandler begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.473 [task_abort_handler.cc:126] [619762]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.487 [api_impl.cc:4192] 619762 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.502 [runtime.cc:4698] 619762 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.553 [adapter_rts.cc:42] [619762]open so: libruntime.so [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.818 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.855 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.872 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.887 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.899 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.915 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.929 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.943 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.202.972 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.202.990 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.001 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.015 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.029 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.047 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.061 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.078 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.091 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.108 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.121 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.139 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.153 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.171 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.186 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.204 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.217 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.234 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.247 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.264 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.278 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.295 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.308 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.326 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.339 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.361 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.374 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.389 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.403 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.417 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.430 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.448 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.462 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.480 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.493 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.511 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.523 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.540 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.553 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.569 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.583 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.599 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.612 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.630 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.644 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.662 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.675 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.696 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.753 [hccl_communicator_attrs.cc:407] [619762]module[0] contains [8]devices [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.771 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.793 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.807 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.821 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.834 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.845 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.858 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.872 [hccl_communicator_attrs.cc:411] [619762]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.885 [hccl_communicator_attrs.cc:425] [619762]different module contains different numbers of cards:[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.898 [hccl_communicator_attrs.cc:426] [619762]different module contains different type of cards:[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.203.958 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.203.983 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.001 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.023 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.039 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.055 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.069 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.087 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.103 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.120 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.132 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.148 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.183 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.204 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.218 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.243 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.257 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.276 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.290 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.309 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.322 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.339 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.352 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.370 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.387 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.405 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.418 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.439 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.450 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.468 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.480 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.495 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.506 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.521 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.536 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.553 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.568 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.587 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.600 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.619 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.641 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.658 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.671 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.687 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.700 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.722 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.734 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.755 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.769 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.786 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.799 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.818 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.833 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.849 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.861 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.879 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.891 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.909 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.923 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.940 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.954 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.204.973 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.204.987 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.004 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.018 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.045 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.060 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.083 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.097 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.117 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.131 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.150 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.166 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.183 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.197 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.213 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.226 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.243 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.255 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.271 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.284 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.300 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.314 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.331 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.344 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.366 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.378 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.395 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.408 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.430 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.443 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.461 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.474 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.494 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.508 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.528 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.540 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.557 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.570 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.585 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.598 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.616 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.629 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.647 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.660 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.677 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.692 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.707 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.721 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.739 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.755 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.773 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.792 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.811 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.833 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.851 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.862 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.879 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.891 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.910 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.923 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.941 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.953 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.205.971 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.205.983 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.003 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.018 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.039 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.049 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.064 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.078 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.094 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.107 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.126 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.141 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.160 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.173 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.192 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.205 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.232 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.245 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.258 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.272 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.287 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.300 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.316 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.332 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.350 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.365 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.383 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.396 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.413 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.424 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.446 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.463 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.484 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.499 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.518 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.530 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.551 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.564 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.583 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.594 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.611 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.632 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.651 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.206.664 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.683 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.727 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.748 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.765 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.783 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.798 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.817 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.832 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.848 [hccl_communicator_attrs.cc:156] [619762][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.872 [hccl_communicator_attrs.cc:1021] [619762][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.892 [hccl_communicator.cc:1440] [619762][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.906 [hccl_communicator.cc:1450] [619762][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.920 [externalinput.cc:1445] [619762]InitParsesProtocol start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.939 [externalinput.cc:1462] [619762]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:48.206.952 [externalinput.cc:1411] [619762]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.206.978 [externalinput.cc:1413] [619762]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.004 [externalinput.cc:1467] [619762]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.053 [config.cc:557] [619762][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.071 [hccl_communicator.cc:668] [619762][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.101 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.122 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.138 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=0. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.158 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=0.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.172 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.190 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.207 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.221 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=1. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.234 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=1.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.245 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.256 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.272 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.285 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=2. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.302 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=2.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.317 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.331 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.345 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.362 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=3. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.376 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=3.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.390 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.404 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.418 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.431 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=5. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.444 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=5.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.466 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.481 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.495 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.509 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=6. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.524 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=6.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.535 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.549 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.563 [api_impl.cc:3520] 619762 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.207.576 [api_impl.cc:3168] 619762 EnableP2P: Enable P2P devIdDes=4, phyIdSrc=7. [INFO] DRV(619435,python3.9):2025-04-03-09:35:48.207.592 [devdrv_pcie.c:888][ascend][curpid:619435,619762][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=4; peer_phy_devid=7.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.606 [p2p_mgmt.cc:74] [619762]enable p2p: local logic id:4, remote physic id:7. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.635 [adapter_hal.cc:298] [619762]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.653 [sal.cc:593] [619762]IsHostUseDevNic[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.699 [adapter_hal.cc:298] [619762]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.720 [sal.cc:593] [619762]IsHostUseDevNic[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.732 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.746 [network_manager.cc:169] [619762]NetworkManager: devicePhyId[4294967295], devicePhyId_[4] deviceLogicId_[4], hasBackup[0] [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.762 [tsd_client.cpp:189][TsdCapabilityGet][tid:619762] TsdCapabilityGet Begin. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.777 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.792 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.802 [process_mode_manager.cpp:1042][CapabilityGet][tid:619762] [ProcessModeManager] enter into CapabilityGet process deviceId[4] type[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.814 [adapter_tdt.cc:66] [619762]Get TsdCapability success. deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.826 [network_manager.cc:175] [619762][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:48.207.843 [network_manager.cc:99] [619762][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.858 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.870 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619762] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.882 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619762] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.893 [process_mode_manager.cpp:386][InitTsdClient][tid:619762] [TsdClient] tsd client has been inited before [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.913 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619762] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619435,python3.9):2025-04-03-09:35:48.207.928 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619762] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.940 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619762] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.954 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619762] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.207.970 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619762] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.208.017 [hdc_client.cpp:154][TsdRecvData][tid:619762] recv timeout is:150000 ms [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.548.178 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.548.184 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.548.208 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.548.220 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.231 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.239 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.251 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.261 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.548.274 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.548.282 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.548.294 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.548.302 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.318 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.548.326 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.932.178 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.932.179 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.932.180 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.287 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.302 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.314 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.326 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.324 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.336 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.373 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.374 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.376 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.384 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.399 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.406 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.415 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.932.421 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.438 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.447 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.450 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.932.467 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.940.774 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.940.803 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.940.817 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:48.940.829 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.964.176 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.964.175 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:48.964.241 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.279 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.297 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.313 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.300 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.325 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.327 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.337 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.344 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.356 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.362 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.370 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.379 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.384 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:48.964.402 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.409 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.418 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.442 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:48.964.449 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.060.174 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.060.185 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.060.196 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.060.211 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.239 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.251 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.262 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.272 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.060.282 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.060.294 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.060.301 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.060.313 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.323 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.060.336 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.572.177 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.572.181 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.572.202 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.572.216 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.227 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.238 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.246 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.259 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.572.270 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.572.282 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.572.288 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.572.302 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.313 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.572.326 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.764.169 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.764.175 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.764.182 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.266 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.298 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.304 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.311 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.321 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.348 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.347 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.355 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.360 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.368 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.374 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.379 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.387 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.764.398 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.399 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.414 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.417 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.764.423 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.796.173 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.796.175 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:49.796.186 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.260 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.291 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.298 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.325 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.328 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.333 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.342 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.353 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.361 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.370 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.377 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.384 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.393 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:49.796.400 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.409 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.425 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.431 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:49.796.441 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.084.181 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.084.187 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.084.204 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.084.221 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.230 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.243 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.251 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.287 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.084.296 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.084.312 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.084.315 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.084.333 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.343 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.084.360 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] TDT(619435,python3.9):2025-04-03-09:35:50.310.430 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619762] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.310.438 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:50.310.459 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619762] [TsdClient] DeviceMsgProc recvMsg realDeviceId[4] msgType[49] localDevId[4] rspCode[0]heterogeneousSubPid[30747], tsdSupportLevel_[63] [INFO] TDT(619435,python3.9):2025-04-03-09:35:50.310.473 [hdc_client.cpp:172][TsdRecvData][tid:619762] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:50.310.486 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619762] OpenSubProc success stype:0, pid:30747 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.310.500 [adapter_tdt.cc:39] [619762]Open TsdProcess success. deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.310.510 [network_manager.cc:118] [619762][TsdProcessOpen]hrtTsdProcessOpen success, subPid[30747], devicePhyId_[4], deviceLogicId_[4], hasBackup[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.310.518 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.310.523 [network_manager.cc:180] [619762][NetworkManager][Init]open tsd by process success, devicePhyId[4], deviceLogicId_[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:50.310.578 [ra_host.c:205]tid:619762,ra_init(205) : Input parameters: phy_id[4], nic_position:[1] hdc_type:[18] [INFO] ROCE(619435,python3.9):2025-04-03-09:35:50.310.649 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619435,python3.9):2025-04-03-09:35:50.310.666 [devdrv_hccl.c:103][ascend][curpid:619435,619762][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619435) [INFO] ROCE(619435,python3.9):2025-04-03-09:35:50.310.703 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:50.310.716 [ra_hdc.c:580]tid:619762,ra_hdc_init(580) : hdc init start! logic id is 4, phy id is 4, hdc_type is 18 [INFO] DRV(619435,python3.9):2025-04-03-09:35:50.310.926 [hdc_core.c:1982][ascend][curpid:619435,619762][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4272; pid=619435) [INFO] HCCP(619435,python3.9):2025-04-03-09:35:50.311.393 [ra_hdc.c:616]tid:619762,ra_hdc_init(616) : hdc init OK! phy_id[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.314.978 [adapter_hccp.cc:583] [619762]init ra success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.314.994 [network_manager.cc:222] [619762]NetworkManager nicDeploy[1] deviceLogicId_[4] devicePhyId_[4] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.024 [hccl_communicator_attrs.cc:1307] [619762][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.036 [hccl_communicator_attrs.cc:1275] [619762]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.062 [hccl_communicator.cc:2259] [619762][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.088 [adapter_hccp.cc:1650] [619762]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.265 [adapter_hccp.cc:2343] [619762]Get available Vnic info success, deviceIdType[0] deviceId[0x4] Vnic ip[192.168.2.195] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.308 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:50.315.326 [ra_host.c:280]tid:619762,ra_socket_init(280) : socket init:mode=1 phy_id=4 family=2 ip=192.168.2.195 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.343 [adapter_hccp.cc:806] [619762]socket init success, ip[3271731392] device id[4], socketHandle[0xfffea00ffbb0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.357 [network_manager.cc:692] [619762]devicePhyId[4], ip[192.168.2.195] socket init OK [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.367 [network_manager.cc:432] [619762][Start][Vnic]trying to listen on ip[192.168.2.195] port[16666]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:50.315.383 [ra_host.c:875]tid:619762,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[4], local_ip[192.168.2.195], port[16666] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.692 [network_manager.cc:448] [619762][Start][Vnic]Listen on ip[192.168.2.195], port[16666] sucess, devPhyId[4], devLogicId[4], isAutoPort[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.710 [network_manager.cc:451] [619762]NetworkManager devicePhyId_[4] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.723 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.736 [hccl_socket.cc:106] [619762][HcclSocket][Listen] device[4] listens on ip[192.168.2.195] port[16666] success, socketType[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.751 [hccl_socket_manager.cc:56] [619762][Init][Server]ip[192.168.2.195] port[16666] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.769 [hccl_communicator.cc:1548] [619762][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.195] port[16666] success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.782 [hccl_communicator_attrs.cc:1307] [619762][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.793 [hccl_communicator_attrs.cc:1275] [619762]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.806 [hccl_communicator.cc:1565] [619762]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.818 [network_manager.cc:69] [619762]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.829 [hccl_communicator.cc:2028] [619762]profiling config information:options[], mode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.315.949 [adapter_trace.cc:21] [619762]Call TraceCreate, return value[0] [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.315.971 [atrace_api.c:61](tid:619762) AtraceCreateWithAttr start. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.316.059 [trace_rb_log.c:45](tid:619762) init ring buffer time finished, realTime 1743644150316052946, monotonicTime 4211952598555, freq 99995 kHz, minutesWest 480 minutes [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.316.088 [trace_rb_log.c:137](tid:619762) [HCCL_619762_4] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.316.105 [tracer_mgr_operate.c:66](tid:619762) create object HCCL_619762_4 success, exitSave(true), noLock(false). [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.316.117 [atrace_api.c:63](tid:619762) AtraceCreateWithAttr end. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.128 [adapter_trace.cc:70] [619762]Call TraceCreateWithAttr, return value[0xaaaaf23809e8], Params: objName[HCCL_619762_4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.186 [rank_consistentcy_checker.cc:32] [619762][GetInstance] get deviceLogicId[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.218 [api_impl.cc:1486] 619762 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.246 [npu_driver.cc:4604] 619762 GetMaxStreamAndTask: deviceId=4, tsId=0, streamCount=1984. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.263 [task_exception_handler.cc:1212] [619762]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.283 [hccl_communicator_attrs.cc:1099] [619762][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.354 [api_impl.cc:4219] 619762 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.377 [command_handle.cc:25] [619762][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.391 [command_handle.cc:99] [619762][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.471 [profiler_manager_impl.cc:76] [619762][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.491 [hccl_communicator.cc:2193] [619762][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.316.503 [hccl_communicator.cc:803] [619762]Will use ffts mode. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.530 [api_impl.cc:3138] 619762 GetDeviceIndexByPhyId: get Index by PhyId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.546 [api_impl.cc:3149] 619762 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.557 [api_impl.cc:3157] 619762 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.316.568 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.317.616 [api_impl.cc:3476] 619762 GetDeviceInfo: get device info, deviceId=4 [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:50.317.662 [tbe_vector_reduce.cc:508] [619762]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.318.752 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.318.924 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.318.941 [dispatcher.cc:157] [619762][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.318.958 [api_impl.cc:3138] 619762 GetDeviceIndexByPhyId: get Index by PhyId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.318.970 [api_impl.cc:3149] 619762 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.318.981 [api_impl.cc:3157] 619762 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.319.001 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.319.032 [api_impl.cc:1990] 619762 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.319.892 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.319.952 [api_impl.cc:1990] 619762 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.319.972 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.320.001 [api_impl.cc:1990] 619762 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.320.017 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.047 [opexecounter.cc:71] [619762]alloc counter mem resource. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.086 [hccl_communicator_attrs.cc:1368] [619762][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.125 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.148 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.185 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.200 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.213 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.225 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.240 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.255 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.270 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.283 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.296 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.309 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.321 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.334 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.346 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.369 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.382 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.394 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.409 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.422 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.435 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.448 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.462 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.474 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.487 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.499 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.511 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.524 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.536 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.549 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.562 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.576 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.588 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.601 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.614 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.628 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.643 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.655 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.674 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.689 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.702 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.714 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.729 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.745 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.758 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.771 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.785 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.799 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.814 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.826 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.838 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.851 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.865 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.877 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.891 [alg_configurator.cc:68] [619762][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.904 [alg_configurator.cc:241] [619762][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.919 [alg_configurator.cc:134] [619762]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.940 [alg_configurator.cc:449] [619762][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.320.954 [alg_configurator.cc:42] [619762]alg configurator init success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.001 [topo_info_extractor.cc:66] [619762]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[4], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.037 [topo_info_extractor.cc:155] [619762][SetRankInfo]rankData_: userRank[4], devicePhyId[4], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.062 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.091 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.121 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.141 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.159 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.177 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.191 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.208 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.243 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.262 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.278 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.295 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.321.312 [api_error.cc:3478] 619762 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.329 [adapter_rts.cc:940] [619762][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.350 [topo_info_extractor.cc:348] [619762]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.483 [topo_info_extractor.cc:888] [619762]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[4], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.515 [topo_info_extractor.cc:585] [619762][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.539 [topo_info_extractor.cc:646] [619762][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.554 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.573 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.589 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.605 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.630 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/1; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.646 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.661 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.676 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.689 [topo_info_extractor.cc:689] [619762]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[4], userRankSize[8], plane size[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.704 [topo_info_extractor.cc:697] [619762][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.719 [topo_info_extractor.cc:734] [619762]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 4/4/0/; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.731 [topo_info_extractor.cc:739] [619762]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[4], userRankSize[8], plane size[1] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.743 [topo_info_extractor.cc:585] [619762][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.759 [topo_info_extractor.cc:646] [619762][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.772 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.794 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.813 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.830 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.847 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/1; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.865 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.880 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.894 [topo_info_extractor.cc:685] [619762]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.907 [topo_info_extractor.cc:689] [619762]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[4], userRankSize[8], plane size[8] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.952 [topo_info_extractor.cc:790] [619762]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId4/4; ] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.321.970 [topo_info_extractor.cc:519] [619762]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.322.104 [api_impl.cc:1990] 619762 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.322.134 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.322.187 [hccl_impl.cc:59] [619762]start hccl resources build:get deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.322.317 [hccl_impl.cc:177] [619762]hcclImpl init success. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:50.322.350 [hccl_aiv.cc:279] [619762][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.664 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.731 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.835 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.852 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.867 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.887 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.325.907 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.006 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.022 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.036 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.050 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.069 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.168 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.182 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.197 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.211 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.232 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.324 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.338 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.364 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.378 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.397 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.486 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.503 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.517 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.532 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.549 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.638 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.653 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.670 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.685 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.702 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.797 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.812 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.825 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.839 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.860 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.947 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.963 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.977 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.326.994 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.013 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.102 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.120 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.135 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.160 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.180 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.271 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.286 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.300 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.314 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.333 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.423 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.438 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.451 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.466 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.483 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.575 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.592 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.607 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.620 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.638 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.726 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.741 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.754 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.771 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.790 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.879 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.898 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.912 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.928 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.327.962 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.050 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.066 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.082 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.097 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.116 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.234 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.256 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.273 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.288 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.307 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.397 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.414 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.428 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.443 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.462 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.549 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.564 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.578 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.592 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.611 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.701 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.716 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.731 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.746 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.766 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.855 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.881 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.898 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.913 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.328.932 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.020 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.035 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.050 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.065 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.084 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.171 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.190 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.205 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.220 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.239 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.331 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.346 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.360 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.375 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.393 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.481 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.497 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.511 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.525 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.544 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.633 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.648 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.671 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.688 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.707 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.795 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.810 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.825 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.842 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.860 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.951 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.966 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.981 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.329.997 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.017 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.104 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.118 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.132 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.145 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.167 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.257 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.272 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.287 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.302 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.320 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.408 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.422 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.435 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.449 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.484 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.577 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.592 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.608 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.622 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.643 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.735 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.751 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.765 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.781 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.804 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.893 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.907 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.923 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.939 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.330.958 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.046 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.061 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.074 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.090 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.109 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.200 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.214 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.228 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.244 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.264 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.359 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.373 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.387 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.403 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.424 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.514 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.529 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.544 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.559 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.579 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.670 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.685 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.700 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.714 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.733 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.825 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.842 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.856 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.873 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.893 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.331.985 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.000 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.014 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.029 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.049 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.139 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.155 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.188 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.205 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.224 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.317 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.333 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.347 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.363 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.382 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.470 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.485 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.501 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.516 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.536 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.625 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.639 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.654 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.669 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.688 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.775 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.791 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.804 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.819 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.839 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.929 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.945 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.960 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.332.984 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.003 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.095 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.110 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.123 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.138 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.156 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.243 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.259 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.274 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.289 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.310 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.401 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.415 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.428 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.442 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.460 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.550 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.566 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.582 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.596 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.614 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.703 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.719 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.731 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.747 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.774 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.864 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.879 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.892 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.907 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.333.925 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.018 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.034 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.048 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.062 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.081 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.169 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.184 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.197 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.211 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.230 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.321 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.337 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.352 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.366 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.386 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.474 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.490 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.502 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.519 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.536 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.626 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.652 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.668 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.686 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.703 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.790 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.805 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.817 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.832 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.851 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.938 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.956 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.972 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.334.988 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.007 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.098 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.114 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.127 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.142 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.161 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.250 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.266 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.280 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.301 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.320 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.408 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.426 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.454 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.470 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.489 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.581 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.596 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.610 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.624 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.644 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.735 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.752 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.767 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.782 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.802 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.888 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.904 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.918 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.934 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.335.954 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.042 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.057 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.073 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.092 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.110 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.207 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.225 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.241 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.259 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.290 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.381 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.397 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.412 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.427 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.447 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.535 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.551 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.566 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.580 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.600 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.687 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.703 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.718 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.736 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.756 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.850 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.866 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.880 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.895 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.336.914 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.002 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.022 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.037 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.054 [runtime.cc:1673] 619762 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.072 [elf.cc:566] 619762 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.169 [runtime.cc:1688] 619762 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.184 [runtime.cc:1953] 619762 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.197 [runtime.cc:1897] 619762 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619435,python3.9):2025-04-03-09:35:50.337.220 [devmm_svm.c:1972][ascend][curpid:619435,619762][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.337.236 [hdc.cc:83] [619762][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.251 [api_impl.cc:1990] 619762 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.277 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.760 [api_impl.cc:1990] 619762 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.782 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.801 [api_impl.cc:2664] 619762 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619435,python3.9):2025-04-03-09:35:50.337.962 [devmm_svm.c:1972][ascend][curpid:619435,619762][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.337.981 [hdc.cc:83] [619762][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.337.993 [api_impl.cc:1990] 619762 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.008 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.030 [api_impl.cc:2096] 619762 HostMalloc: size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.047 [api_error.cc:1260] 619762 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.324 [api_impl.cc:1990] 619762 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.405 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.425 [api_impl.cc:2664] 619762 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.561 [api_impl.cc:1990] 619762 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.582 [api_error.cc:1218] 619762 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.598 [api_impl.cc:2664] 619762 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.626 [peterson_lock.cc:69] [619762][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.639 [peterson_lock.cc:47] [619762][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.669 [hccl_comm.cc:129] [619762]hcclCommInitInfo:commId[hccl-619435-0-1743644148-hccl_world_group], rank[4], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[4], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.703 [hccl_comm.cc:175] [619762][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.724 [topo_matcher.cc:528] [619762][SetAivMode]AivMode is set to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.738 [topo_matcher.cc:540] [619762][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.757 [hcom_plugin.cc:349] [619762][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.338.785 [runtime.cc:5239] 619762 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.800 [adapter_rts.cc:198] [619762][hrtGetDeviceRefresh]deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.813 [hcom_common.cc:85] [619762][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.832 [op_base.cc:128] [619762][HcclGetDeviceId] deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.845 [hccl_comm.cc:1296] [619762]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffeb27cccb8] topoSize[2] [INFO] GE(619435,python3.9):2025-04-03-09:35:50.338.868 [hcom_topo_info.cc:24]619762 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.880 [hcom_plugin.cc:371] [619762][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.904 [topoinfo_detect.cc:555] [619762]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.917 [topoinfo_detect.cc:321] [619762]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.929 [topoinfo_detect.cc:219] [619762]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.951 [op_base.cc:1050] [619762][InitCommRootInfo]Init success, rankNum[8], rank[4], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.967 [profiler_base.cc:342] [619762]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.338.983 [op_base.cc:1107] [619762][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4977843]us, rankNum[8], rank[4] [WARNING] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.339.041 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.339.065 [op_base.cc:853] [619762]HcclGetCommName input handle=0xfffea0022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.339.093 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 4 [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.339.112 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 4, group name: hccl_world_group [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.339.208 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.339.227 [device.cpp:183]619762 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.339.259 [api_impl.cc:3419] 619762 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.339.272 [runtime.cc:3444] 619762 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] DEVICE(619435,fffeb17aefa0,python3.9):2025-04-03-09:35:50.339.267 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 4, group name: hccl_world_group [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.339.287 [api_impl.cc:3449] 619762 DeviceReset: Succ, userDeviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.339.303 [device.cpp:189]619762 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 4. [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.339.322 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:50.339.393 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5031.93 msec. [WARNING] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:50.339.427 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.339.500 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5533.34 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.339.539 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5533.41 msec. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.339.556 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.339.576 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9691.87 msec. [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.339.603 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.353.960 [device.cpp:143]619435 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.003 [api_impl.cc:2999] 619435 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.017 [device.cc:306] 619435 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.033 [runtime.cc:3393] 619435 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.049 [runtime.cc:3560] 619435 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.062 [api_impl.cc:2987] 619435 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.075 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.105 [api_impl.cc:2992] 619435 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.123 [api_impl.cc:3025] 619435 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.137 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.150 [acl.cpp:121]619435 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.162 [device.cpp:154]619435 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.354.212 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.235 [context.cpp:174]619435 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.252 [api_impl.cc:6913] 619435 CtxSetSysParamOpt: curCtx = 0xaaab02fde2d0, configOpt=0, configVal=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.265 [context.cpp:158]619435 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.354.839 [op_base.cc:1214] [619435][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.354.861 [op_base.cc:128] [619435][HcclGetDeviceId] deviceLogicId[4] [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.882 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.354.897 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.354.914 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.354.952 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.221 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.522 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.576 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.667 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 91.14 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.700 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.743 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 40.74 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.767 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.786 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 19.36 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.804 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.860 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 52.76 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.893 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.374 msec. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.355.974 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.022 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 5.67 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.084 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 35.96 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.123 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.147 msec. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.149 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.200 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.248 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 45.171 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.269 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.301 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 29.57 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.323 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.341 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 16.61 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.356 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.388 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 28.89 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.432 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.279 msec. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.465 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.539 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.589 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 47.7 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.655 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 29.7 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.699 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 18.6 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.746 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 23.56 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.787 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 18.51 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.884 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 73.03 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.911 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.96 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.930 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.356.985 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 33.55 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.033 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 23.65 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.052 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.103 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 27.75 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.197 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 70.6 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.219 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.268 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 30.76 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.311 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 0.842 msec. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.330 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.377 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 1.402 msec. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.727 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 34.23 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.755 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.841 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 82.55 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.861 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.895 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 30.49 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.917 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.949 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 26.95 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.973 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.357.992 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 18.93 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.018 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 7.27 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.044 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.062 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 21.5 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.082 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.101 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 17.87 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.116 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.133 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 15.77 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.150 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.169 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 19.34 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.186 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.203 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 15.98 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.219 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.234 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 14.07 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.264 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.303 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 36.091 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.325 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.344 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 20.26 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.363 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.379 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 15.19 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.394 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.411 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 15.98 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.429 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.445 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 15.21 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.463 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.477 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 14.34 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.495 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.511 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 16.34 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.527 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 56.96 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.612 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.630 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 18.77 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.649 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.666 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 15.35 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.682 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.707 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 22.6 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.723 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.738 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 14.06 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.756 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.772 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 15.01 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.789 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.808 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 19.7 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.824 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.840 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 16.03 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.856 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.870 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 12.52 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.887 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.904 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 16.08 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.358.971 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 48.471 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 134.18 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.159 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.210 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 31.31 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.229 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.277 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 29.16 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.294 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.349 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 24.24 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.369 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.397 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 9.86 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.415 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.438 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 6.55 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.455 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.504 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 30.44 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.524 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.570 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 28.64 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.591 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.669 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 56.411 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.691 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.780 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 69.11 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.803 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.823 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.945 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 122.15 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.968 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.359.986 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.122 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 134.15 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.142 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.239 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 79.31 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.261 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.336 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 55.171 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.355 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.373 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.390 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.404 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.418 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.446 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.460 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.472 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.486 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.502 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.517 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.558 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.574 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.814 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 217.31 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.840 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.855 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.870 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.884 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.898 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.360.992 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 73.81 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.017 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.032 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.050 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.105 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 37.74 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.155 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 3.731 msec. [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.463 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.566 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.713 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 88.031 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 28.89 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.929 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 55.72 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.361.968 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 12.37 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.015 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 25.36 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.117 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 79.07 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 26.43 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.213 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 22.57 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.272 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 37.84 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.299 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 6.32 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.341 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.566 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.390 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 0.823 msec. [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.441 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.746 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.781 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.354 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.799 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.817 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.362.916 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 39.56 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.000 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 55.57 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.069 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 41.291 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.158 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 63.38 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.195 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.212 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.272 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 36.76 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.305 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 12.35 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.326 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 0.7 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.365 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.529 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.413 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.576 msec. [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.450 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.553 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.579 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 0.83 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.624 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 21.48 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.652 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 3.9 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.673 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.76 us [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.702 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 82.221 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.796 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 0.69 us [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.822 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.292 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.869 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.433 msec. [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.905 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.363.983 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.087 msec. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.364.265 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.364.302 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.364.318 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.354 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.598 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.642 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.727 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.827 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.11 msec. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.881 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.919 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.941 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.960 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.976 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.364.992 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.365.035 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.365.280 [context.cpp:72]619789 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.365.313 [api_impl.cc:3717] 619789 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.365.334 [awatchdog_core.c:175](tid:619789) create watchdog for id : 65543, tid : 619789, timeout : 300s [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.365.348 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.365.348 [context.cpp:80]619789 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.365.412 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.365.470 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.365.492 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.366.801 [api_impl.cc:7115] 619789 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5a00000 [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.376.914 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.376.958 [api_impl.cc:7129] 619789 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.073 [api_impl.cc:7145] 619789 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.147 [device.cpp:143]619789 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.166 [api_impl.cc:2999] 619789 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.178 [device.cc:306] 619789 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.191 [runtime.cc:3393] 619789 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.205 [runtime.cc:3560] 619789 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.217 [api_impl.cc:2987] 619789 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.227 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.241 [api_impl.cc:2992] 619789 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.260 [api_impl.cc:3025] 619789 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.273 [acl.cpp:115]619789 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.285 [acl.cpp:121]619789 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.297 [device.cpp:154]619789 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.377.326 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.342 [context.cpp:174]619789 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.357 [api_impl.cc:6913] 619789 CtxSetSysParamOpt: curCtx = 0xaaab02fde2d0, configOpt=0, configVal=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.370 [context.cpp:158]619789 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.377.432 [op_base.cc:1214] [619789][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.450 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.377.463 [adapter_rts.cc:234] [619789][hrtGetDevice]deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:50.377.483 [op_base.cc:128] [619789][HcclGetDeviceId] deviceLogicId[4] [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.513 [context.cpp:72]619789 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.528 [api_impl.cc:3717] 619789 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.541 [context.cpp:80]619789 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.560 [api_error.cc:1807] 619789 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.574 [memory_task.cc:342] 619789 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.588 [memory_task.cc:473] 619789 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.598 [memory_task.cc:518] 619789 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.612 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.628 [memory_task.cc:1007] 619789 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaab0eb04e90, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.641 [memory_task.cc:1166] 619789 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.652 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.665 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.682 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.695 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.712 [callback.cpp:97]619789 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.739 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.754 [task_info.cc:521] 619789 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.766 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.779 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.793 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.806 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.815 [callback.cpp:109]619789 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.876 [npu_driver.cc:3844] 619606 CqReportGet: cq report get start, device_id=4, tsId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.901 [npu_driver.cc:3851] 619606 CqReportGet: halCqReportGet count=1 [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.377.900 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.914 [api_impl.cc:5087] 619606 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.377.938 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.939 [callback.cpp:138]619606 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.377.960 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.377.960 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.377.983 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.379.245 [api_impl.cc:7115] 619789 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12d100000000, end ptr=0x12df35a00000 [INFO] PRE_ACT(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.386.249 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.281 [api_impl.cc:7129] 619789 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.371 [api_impl.cc:7145] 619789 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.386.521 [device.cpp:231]619789 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.539 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.386.555 [acl.cpp:802]619789 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.662 [npu_driver.cc:404] 619789 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.703 [stream.cc:2246] 619789 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.736 [stream_sqcq_manage.cc:147] 619789 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.386.749 [npu_driver.cc:882] 619789 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.182 [npu_driver.cc:6125] 619789 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.231 [npu_driver.cc:6137] 619789 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=324, streamId=6, logicCq=324, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.250 [npu_driver.cc:6189] 619789 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=6, logicCq=324. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.264 [npu_driver.cc:5916] 619789 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e002f000, addr=dfffe002f000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.285 [stream.cc:653] 619789 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.321 [stream.cc:276] 619789 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.335 [stream.cc:688] 619789 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=4, streamResId=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.387.350 [acl_resource_manager.cpp:814]619789 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.387.361 [acl_resource_manager.cpp:816]619789 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.499 [engine.cc:188] 619789 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.516 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.531 [task_info.cc:409] 619789 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.542 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.554 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.570 [npu_driver.cc:751] 619789 SqTaskSend: device_id=4, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.581 [stars_engine.cc:893] 619789 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.594 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.609 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.621 [stream.cc:2886] 619789 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.633 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.644 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.654 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.666 [logger.cc:152] 619789 TaskFinished: device_id=4, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.682 [stars_engine.cc:437] 619789 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.695 [stream.cc:2886] 619789 StarsWaitForTask: No.1 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.706 [stream.cc:852] 619789 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.716 [stream.cc:938] 619789 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.726 [stream.cc:1507] 619789 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.745 [stream_sqcq_manage.cc:451] 619789 FreeLogicCq: Return(0), threadIdentifier(281469381308320), devId(4), tsId(0), cqId(324), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.764 [npu_driver.cc:6163] 619789 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=6, logicCq=324. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.803 [npu_driver.cc:1180] 619789 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=324, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.817 [stream.cc:237] 619789 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.829 [stream_sqcq_manage.cc:216] 619789 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.387.838 [npu_driver.cc:966] 619789 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.388.425 [npu_driver.cc:974] 619789 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.388.442 [stream_sqcq_manage.cc:228] 619789 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.388.454 [npu_driver.cc:432] 619789 StreamIdFree: stream id free begin, device_id=4, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.388.470 [npu_driver.cc:441] 619789 StreamIdFree: Stream id free success, stream_id=6, device_id=4, tsId=0, streamFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.388.485 [device_msg_handler.cc:110] 619789 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.389.186 [device_msg_handler.cc:127] 619789 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.389.236 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.389.272 [npu_driver.cc:500] 619789 GetFaultEvent: get fault event, device_id=4, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.389.343 [acl.cpp:782]619789 GetFaultEventInfo: can not get fault event of device 4, runtime errorCode is 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:50.389.364 [error_manager.cc:989]619789 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.389.669 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.389.706 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.389.724 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.389.738 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.389.917 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.027 [context.cpp:72]619789 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.046 [api_impl.cc:3717] 619789 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.060 [context.cpp:80]619789 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.080 [api_error.cc:1807] 619789 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.112 [memory_task.cc:342] 619789 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.126 [memory_task.cc:473] 619789 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.136 [memory_task.cc:518] 619789 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.148 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.163 [memory_task.cc:1007] 619789 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaab0eb058b0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.175 [memory_task.cc:1166] 619789 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.185 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.197 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=4 [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.390.185 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.218 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.239 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.257 [callback.cpp:97]619789 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.272 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.284 [task_info.cc:521] 619789 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.298 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.310 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.325 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.338 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.348 [callback.cpp:109]619789 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.367 [npu_driver.cc:3844] 619606 CqReportGet: cq report get start, device_id=4, tsId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.388 [npu_driver.cc:3851] 619606 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.404 [api_impl.cc:5087] 619606 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.418 [device.cpp:231]619789 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.423 [callback.cpp:138]619606 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.439 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.460 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.390.470 [acl.cpp:802]619789 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.483 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.507 [npu_driver.cc:404] 619789 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.536 [stream.cc:2246] 619789 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.572 [stream_sqcq_manage.cc:147] 619789 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.390.588 [npu_driver.cc:882] 619789 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.008 [npu_driver.cc:6125] 619789 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.052 [npu_driver.cc:6137] 619789 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=325, streamId=7, logicCq=325, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.073 [npu_driver.cc:6189] 619789 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=7, logicCq=325. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.088 [npu_driver.cc:5916] 619789 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e002f000, addr=dfffe002f000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.100 [stream.cc:653] 619789 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.135 [stream.cc:276] 619789 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.151 [stream.cc:688] 619789 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=4, streamResId=65534 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.391.167 [acl_resource_manager.cpp:814]619789 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.391.178 [acl_resource_manager.cpp:816]619789 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.216 [engine.cc:188] 619789 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.234 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.247 [task_info.cc:409] 619789 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.260 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.271 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.285 [npu_driver.cc:751] 619789 SqTaskSend: device_id=4, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.297 [stars_engine.cc:893] 619789 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.311 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.337 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.348 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.359 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.369 [logger.cc:152] 619789 TaskFinished: device_id=4, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.385 [stars_engine.cc:437] 619789 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.399 [stream.cc:2886] 619789 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.413 [stream.cc:852] 619789 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.425 [stream.cc:938] 619789 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.436 [stream.cc:1507] 619789 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.457 [stream_sqcq_manage.cc:451] 619789 FreeLogicCq: Return(0), threadIdentifier(281469381308320), devId(4), tsId(0), cqId(325), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.477 [npu_driver.cc:6163] 619789 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=7, logicCq=325. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.503 [npu_driver.cc:1180] 619789 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=325, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.514 [stream.cc:237] 619789 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.526 [stream_sqcq_manage.cc:216] 619789 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.391.536 [npu_driver.cc:966] 619789 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.000 [npu_driver.cc:974] 619789 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.019 [stream_sqcq_manage.cc:228] 619789 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.034 [npu_driver.cc:432] 619789 StreamIdFree: stream id free begin, device_id=4, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.052 [npu_driver.cc:441] 619789 StreamIdFree: Stream id free success, stream_id=7, device_id=4, tsId=0, streamFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.067 [device_msg_handler.cc:110] 619789 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.385 [device_msg_handler.cc:127] 619789 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.419 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.392.441 [npu_driver.cc:500] 619789 GetFaultEvent: get fault event, device_id=4, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.392.499 [acl.cpp:782]619789 GetFaultEventInfo: can not get fault event of device 4, runtime errorCode is 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:50.392.514 [error_manager.cc:989]619789 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.392.802 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.392.847 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.392.866 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.392.879 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.392.947 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.023 [context.cpp:72]619789 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.042 [api_impl.cc:3717] 619789 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.057 [context.cpp:80]619789 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.074 [api_error.cc:1807] 619789 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.091 [memory_task.cc:342] 619789 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.103 [memory_task.cc:473] 619789 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.114 [memory_task.cc:518] 619789 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.126 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.143 [memory_task.cc:1007] 619789 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaab0eb06320, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.156 [memory_task.cc:1166] 619789 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.167 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.179 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.194 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.209 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.226 [callback.cpp:97]619789 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.244 [task_res.cc:380] 619789 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.260 [task_info.cc:521] 619789 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.271 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.295 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.311 [task_submit.cc:573] 619789 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.325 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.338 [callback.cpp:109]619789 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.352 [npu_driver.cc:3844] 619606 CqReportGet: cq report get start, device_id=4, tsId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.372 [npu_driver.cc:3851] 619606 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.382 [api_impl.cc:5087] 619606 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.399 [callback.cpp:138]619606 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.403 [device.cpp:231]619789 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.415 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.426 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.434 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.443 [acl.cpp:802]619789 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.479 [npu_driver.cc:404] 619789 StreamIdAlloc: stream id alloc begin, device_id=4, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.508 [stream.cc:2246] 619789 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.543 [stream_sqcq_manage.cc:147] 619789 Alloc: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.559 [npu_driver.cc:882] 619789 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.831 [npu_driver.cc:6125] 619789 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.875 [npu_driver.cc:6137] 619789 LogicCqAllocateV2: success: deviceId=4, tsId=0, cqId=326, streamId=8, logicCq=326, cqeDepth=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.896 [npu_driver.cc:6189] 619789 StreamBindLogicCq: success: deviceId=4, tsId=0, streamId=8, logicCq=326. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.911 [npu_driver.cc:5916] 619789 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e002f000, addr=dfffe002f000size=4096 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.924 [stream.cc:653] 619789 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.958 [stream.cc:276] 619789 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.393.974 [stream.cc:688] 619789 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=4, streamResId=65534 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.393.988 [acl_resource_manager.cpp:814]619789 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.394.006 [acl_resource_manager.cpp:816]619789 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.046 [engine.cc:188] 619789 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.063 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.077 [task_info.cc:409] 619789 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.088 [stream.cc:1400] 619789 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.100 [stream.cc:2595] 619789 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.115 [npu_driver.cc:751] 619789 SqTaskSend: device_id=4, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.129 [stars_engine.cc:893] 619789 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.140 [stream.cc:1494] 619789 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.153 [npu_driver.cc:5547] 619789 GetSqHead: deviceId=4, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.168 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.178 [task.cc:185] 619789 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.190 [logger.cc:152] 619789 TaskFinished: device_id=4, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.204 [stars_engine.cc:437] 619789 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.219 [stream.cc:2886] 619789 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.230 [stream.cc:852] 619789 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.243 [stream.cc:938] 619789 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.254 [stream.cc:1507] 619789 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.272 [stream_sqcq_manage.cc:451] 619789 FreeLogicCq: Return(0), threadIdentifier(281469381308320), devId(4), tsId(0), cqId(326), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.294 [npu_driver.cc:6163] 619789 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=8, logicCq=326. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.324 [npu_driver.cc:1180] 619789 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=326, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.339 [stream.cc:237] 619789 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.353 [stream_sqcq_manage.cc:216] 619789 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.364 [npu_driver.cc:966] 619789 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.833 [npu_driver.cc:974] 619789 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.852 [stream_sqcq_manage.cc:228] 619789 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.881 [npu_driver.cc:432] 619789 StreamIdFree: stream id free begin, device_id=4, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.898 [npu_driver.cc:441] 619789 StreamIdFree: Stream id free success, stream_id=8, device_id=4, tsId=0, streamFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.394.912 [device_msg_handler.cc:110] 619789 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.395.212 [device_msg_handler.cc:127] 619789 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.395.243 [runtime.cc:5239] 619789 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.395.265 [npu_driver.cc:500] 619789 GetFaultEvent: get fault event, device_id=4, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.395.327 [acl.cpp:782]619789 GetFaultEventInfo: can not get fault event of device 4, runtime errorCode is 0 [INFO] GE(619435,python3.9):2025-04-03-09:35:50.395.346 [error_manager.cc:989]619789 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.395.632 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.395.663 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.395.680 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.395.696 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619435,fffeb27cefa0,python3.9):2025-04-03-09:35:50.395.760 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:50.469.086 [awatchdog_monitor.c:160](tid:619468) thread 619762 not exist [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.596.177 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.596.177 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.596.179 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.596.191 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.596.249 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.268 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.289 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.293 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.314 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.328 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.338 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.354 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.361 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.368 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.380 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.381 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.392 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.393 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.408 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.414 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.428 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.442 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.454 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.547 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.581 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.596.598 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.614 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.596.632 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.628.313 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.628.340 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.628.363 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.454 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.499 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.503 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.510 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.556 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.570 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.576 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.563 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.586 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.598 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.605 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.614 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.624 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.628.632 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.654 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.661 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.675 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.628.667 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.916.189 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:50.916.227 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.916.245 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:50.916.261 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.916.276 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.916.291 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.916.331 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.972.958 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.007 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.973.045 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.973.090 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.111 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.131 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.207 [stars_engine.cc:643] 619435 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.254 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.272 [task_manager.cc:485] 619435 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.289 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.308 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.323 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.343 [stars_engine.cc:643] 619435 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.359 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.373 [task_manager.cc:485] 619435 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.384 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.398 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.412 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.431 [stars_engine.cc:643] 619435 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.446 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.460 [task_manager.cc:485] 619435 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.473 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.489 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.503 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.540 [stars_engine.cc:622] 619435 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=320, retCode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.557 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.574 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.973.589 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.973.610 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.641 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.657 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.685 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.701 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.714 [stream.cc:2886] 619435 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.728 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.739 [stream.cc:2886] 619435 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.750 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.764 [stream.cc:2886] 619435 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.778 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.793 [stream.cc:2886] 619435 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.805 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.817 [stream.cc:2886] 619435 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.833 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.843 [stream.cc:2886] 619435 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.854 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.867 [stream.cc:2886] 619435 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.879 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.892 [stream.cc:2886] 619435 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.905 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.918 [stream.cc:2886] 619435 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.931 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.944 [stream.cc:2886] 619435 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.973.964 [event.cc:310] 619435 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.000 [task_res.cc:380] 619435 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=4, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.025 [event.cc:249] 619435 UpdateLatestRecord: device_id=4, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.040 [event_task.cc:259] 619435 ConstructSqeForEventRecordTask: event_record: device_id=4, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.053 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.066 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.094 [task_submit.cc:573] 619435 AllocTaskAndSendStars: device_id=4, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.109 [stars_engine.cc:1122] 619435 SyncTask: Task Wait: device_id=4, stream_id=3, task_id=0, logicCqId=321. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.127 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.139 [stars_engine.cc:1071] 619435 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.154 [stars_engine.cc:1017] 619435 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.170 [stars_engine.cc:1027] 619435 ProcReport: taskType=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.182 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.196 [event.cc:249] 619435 UpdateLatestRecord: device_id=4, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.210 [event_task.cc:211] 619435 DoCompleteSuccessForEventRecordTask: event_record: device_id=4, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea5710ce35 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.225 [event.cc:122] 619435 TryFreeEventIdAndCheckCanBeDelete: device_id=4, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.239 [task_res.cc:323] 619435 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=4, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.261 [npu_driver.cc:5618] 619435 GetSqEnable: deviceId=4, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.274 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.285 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.296 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.332 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.347 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.365 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.378 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.391 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.403 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.415 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.427 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.437 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.450 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.464 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.476 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.503 [stars_engine.cc:622] 619435 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=321, retCode=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.515 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:50.974.527 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.538 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:50.974.553 [memory.cpp:421]619435 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.57s ============================================================================================================== 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340876] [debug] [619762] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340885] [debug] [619762] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340892] [debug] [619762] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340900] [debug] [619762] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360664] [debug] [619762] [lcal_comm.cpp:254] InitCommMem 4/8, localRank_ : 4, localRankSize_ : 8 success [2025-04-03 09:35:45.360778] [info] [619762] [lcal_comm.cpp:259] LcalCommInit 4/8 success and extraFlag:0 commArgs_.localRank : 4 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.386491] [error] [619789] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.390395] [error] [619789] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.393381] [error] [619789] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.146 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.187 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:50.983.260 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619435:281473359609888,MainProcess):2025-04-03-09:35:50.983.426 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.504 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.538 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.578 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:50.983.597 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.108.209 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.108.245 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.108.264 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.108.283 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.108.308 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.108.328 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.108.365 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.410 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.514 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619435,ffff3f07afa0,python3.9):2025-04-03-09:35:51.151.610 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.701 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.718 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619435,ffff3f88afa0,python3.9):2025-04-03-09:35:51.151.777 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.851 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.867 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.151.921 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619435,ffff3e05afa0,python3.9):2025-04-03-09:35:51.152.007 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.139 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.167 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619435,ffff3e86afa0,python3.9):2025-04-03-09:35:51.152.211 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.300 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.317 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.334 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.350 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.464 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.489 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.505 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.534 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.550 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.567 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.581 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.605 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.617 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.631 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.645 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.658 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.671 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.683 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.695 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.713 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.730 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.742 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.755 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.767 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.778 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.788 [stream.cpp:131]619435 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.803 [api_impl.cc:1394] 619435 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.814 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.824 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.152.836 [api_error.cc:895] 619435 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.152.846 [stream.cpp:146]619435 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.891 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.936 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.966 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.980 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.152.993 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.130 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.147 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.700 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.723 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.738 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.752 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.766 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.153.995 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.154.016 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.154.032 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.154.096 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.154.112 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.154.127 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_4 [INFO] GE_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.160.549 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.160.585 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.160.605 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.619 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.645 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.662 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.674 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.685 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.695 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.717 [device.cpp:143]619435 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.734 [api_impl.cc:2999] 619435 SetDevice: device_id=4, deviceMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.746 [device.cc:306] 619435 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.759 [runtime.cc:3393] 619435 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.773 [runtime.cc:3560] 619435 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.801 [api_impl.cc:2987] 619435 NewDevice: SetCurRef = 0xaaab02fde2d0,userDeviceId=4,deviceId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.816 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.835 [api_impl.cc:2992] 619435 NewDevice: New device success, user_device_id=4, die_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.160.850 [api_impl.cc:3025] 619435 SetDevice: SetDevice success, curCtx=0xaaab02fde2d0, user_device_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.863 [acl.cpp:115]619435 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.875 [acl.cpp:121]619435 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.160.888 [device.cpp:154]619435 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 4 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:51.160.911 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:51.239.196 [awatchdog_monitor.c:160](tid:619468) thread 619789 not exist [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.428.180 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.428.192 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.428.194 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.428.237 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.254 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.269 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.283 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.290 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.296 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.313 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.317 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.324 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.333 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.341 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.428.342 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.355 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.361 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.395 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.399 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.407 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.413 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.426 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.508 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.531 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.552 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.428.568 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.581 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.428.594 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.460.171 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.460.172 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.460.177 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.291 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.295 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.329 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.332 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.336 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.344 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.358 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.364 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.370 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.383 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.389 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.413 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.460.425 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.435 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.442 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.454 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.461 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.460.477 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.532.648 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:51.532.680 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:51.532.697 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:51.532.709 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.532.732 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.532.819 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.620.183 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.620.210 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.620.228 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.620.243 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.620.261 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.620.279 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.620.297 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.940.173 [trs_cb_event.c:119][ascend][curpid:619435,619606][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =320; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:51.940.209 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.940.225 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:51.940.255 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.940.270 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:51.940.284 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:51.940.301 [npu_driver.cc:3828] 619606 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=320. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.132.183 [trs_cb_event.c:119][ascend][curpid:619435,619607][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=4, tid =321; ret=16) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.132.212 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.132.230 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.132.246 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.132.259 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.132.275 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.132.292 [npu_driver.cc:3828] 619607 CqReportIrqWait: Irq wait start, deviceId=4, tsId=0, groupId=321. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.260.168 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.260.176 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.260.189 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.281 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.284 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.298 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.304 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.310 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.316 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.321 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.324 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.327 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.332 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.343 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.350 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.390 [tensor_data_transfer.cpp:1054]619715 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.353 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.405 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.260.412 [tensor_data_transfer.cpp:1056]619715 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.434 [api_impl.cc:6121] 619715 MemQueuePeek: Start to peek queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.260.448 [npu_driver.cc:4330] 619715 MemQueuePeek: peek mem queue, deviceId=4, qid=0, timeout=800. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.292.176 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.292.184 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.292.185 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.285 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.303 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.293 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.315 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.318 [tensor_data_transfer.cpp:1054]619723 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_scalar_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.326 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.334 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.339 [tensor_data_transfer.cpp:1056]619723 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.347 [tensor_data_transfer.cpp:1054]619721 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_image_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.353 [tensor_data_transfer.cpp:1054]619725 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.359 [api_impl.cc:6121] 619723 MemQueuePeek: Start to peek queue on devId 4, qid is 4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.365 [tensor_data_transfer.cpp:1056]619721 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.292.370 [tensor_data_transfer.cpp:1056]619725 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.376 [npu_driver.cc:4330] 619723 MemQueuePeek: peek mem queue, deviceId=4, qid=4, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.381 [api_impl.cc:6121] 619721 MemQueuePeek: Start to peek queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.384 [api_impl.cc:6121] 619725 MemQueuePeek: Start to peek queue on devId 4, qid is 5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.405 [npu_driver.cc:4330] 619721 MemQueuePeek: peek mem queue, deviceId=4, qid=3, timeout=800. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.292.410 [npu_driver.cc:4330] 619725 MemQueuePeek: peek mem queue, deviceId=4, qid=5, timeout=800. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.278 [op_base.cc:2136] [619435]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619435,fffeb17aefa0,python3.9):2025-04-03-09:35:52.340.298 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.340.305 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.326 [adapter_rts.cc:198] [619435][hrtGetDeviceRefresh]deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.344 [op_base.cc:2151] [619435][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.365 [op_base.cc:128] [619435][HcclGetDeviceId] deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.400 [op_base.cc:2168] [619435]com is not global com [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.419 [hcom_plugin.cc:381] [619435][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.472 [hccl_comm.cc:1375] [619435]UnRegistTaskAbortHandler begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.488 [task_abort_handler.cc:140] [619435]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.340.503 [api_impl.cc:4192] 619435 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.340.516 [runtime.cc:4698] 619435 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.532 [hccl_communicator.cc:2112] [619435][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.340.547 [hccl_communicator.cc:2114] [619435][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.558 [hccl_communicator.cc:2082] [619435]start to stop the backGround Thread [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.340.572 [heartbeat.cc:1203] [619435]Heartbeat has been destroyed [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.584 [heartbeat.cc:451] [619435][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.340.595 [heartbeat.cc:452] [619435]Heartbeat has been destroyed [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.611 [hccl_impl.cc:69] [619435]start hccl resources destruction:deviceLogicId[4] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.340.628 [heartbeat.cc:1203] [619435]Heartbeat has been destroyed [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.639 [heartbeat.cc:451] [619435][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.340.652 [heartbeat.cc:452] [619435]Heartbeat has been destroyed [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.340.753 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.340.911 [task_exception_handler.cc:1232] [619435]deInit taskFailCallback [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.341.574 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.341.609 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.341.654 [api_error.cc:1278] 619435 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.341.670 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.341.706 [hccl_socket_manager.cc:81] [619435][DeInit][Server]ip[192.168.2.195] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.341.724 [hccl_socket_manager.cc:83] [619435][DeInit][Server]ip[192.168.2.195] port[16666] [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.341.736 [hccl_socket.cc:232] [619435][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.341.749 [network_manager.cc:69] [619435]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.341.760 [network_manager.cc:458] [619435]NetworkManager devicePhyId_[4] vnic stopped. refcount[0] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:52.341.782 [ra_host.c:911]tid:619435,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[4], local_ip[192.168.2.195] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:52.342.139 [ra_host.c:404]tid:619435,ra_socket_deinit(404) : Input parameters: phy_id[4] family[2] local_ip[192.168.2.195] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.342.293 [hccl_socket.cc:68] [619435][HcclSocket][DeInit] device[4] stops listen on ip[192.168.2.195], port[16666] success, socketType[0]. [WARNING] HCCL(619435,python3.9):2025-04-03-09:35:52.342.313 [hccl_socket.cc:232] [619435][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.342.338 [hccl_communicator.cc:1741] [619435][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.342.354 [network_manager.cc:69] [619435]NetworkManager::GetInstance deviceLogicID[4]. [INFO] HCCP(619435,python3.9):2025-04-03-09:35:52.342.366 [ra_host.c:248]tid:619435,ra_deinit(248) : Input parameters: phy_id[4], nic_position:[1] [INFO] HCCP(619435,python3.9):2025-04-03-09:35:52.342.381 [ra_hdc.c:651]tid:619435,ra_hdc_deinit(651) : hdc deinit start! phy_id[4] [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.342.479 [hdc_client.c:417][ascend][curpid:619435,619435][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4272) [INFO] ROCE(619435,python3.9):2025-04-03-09:35:52.342.542 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619435,python3.9):2025-04-03-09:35:52.342.559 [ra_hdc.c:685]tid:619435,ra_hdc_deinit(685) : hdc deinit OK! phy_id[4] [INFO] ROCE(619435,python3.9):2025-04-03-09:35:52.342.572 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.636 [tsd_client.cpp:189][TsdCapabilityGet][tid:619435] TsdCapabilityGet Begin. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.651 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.662 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.675 [process_mode_manager.cpp:1042][CapabilityGet][tid:619435] [ProcessModeManager] enter into CapabilityGet process deviceId[4] type[6] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.342.689 [adapter_tdt.cc:66] [619435]Get TsdCapability success. deviceLogicId[4] [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.702 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.714 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.734 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619435] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.751 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619435] add close subproc:30747, proctype:0 [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.342.795 [hdc_client.cpp:154][TsdRecvData][tid:619435] recv timeout is:150000 ms [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.414.506 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619435] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.414.524 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619435] [TsdClient] DeviceMsgProc recvMsg realDeviceId[4] msgType[61] localDevId[4] rspCode[0]heterogeneousSubPid[30747], tsdSupportLevel_[63] [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.414.539 [hdc_client.cpp:172][TsdRecvData][tid:619435] [deviceId=4] the Tsd recv data already [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.414.550 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619435] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.414.563 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619435] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.574 [adapter_tdt.cc:52] [619435]Open TsdProcess success. deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.585 [network_manager.cc:405] [619435][DeInit]finish CloseHccpProcess, phy_id[4], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.598 [network_manager.cc:409] [619435]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.610 [hccl_communicator_attrs.cc:1307] [619435][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.622 [hccl_communicator_attrs.cc:1275] [619435]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.639 [notify_pool_impl.cc:43] [619435]NotifyPoolImpl Destroy. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.661 [notify_pool_impl.cc:56] [619435]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.682 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.693 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.703 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.718 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.728 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.738 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.749 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.762 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.783 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.796 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.809 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=0. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.414.829 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=0.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.849 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.859 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.870 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.879 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.891 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.901 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.913 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.925 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=1. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.414.939 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=1.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.953 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.965 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.414.977 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.414.990 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.002 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.012 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.023 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.036 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=2. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.415.050 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=2.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.064 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.075 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.085 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.099 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.109 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.119 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.131 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.143 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=3. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.415.158 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=3.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.171 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.188 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.199 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.211 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.224 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.235 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.245 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.256 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.266 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.280 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.294 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.307 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=5. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.415.319 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=5.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.333 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.347 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.358 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.369 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.381 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.397 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.412 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.425 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=6. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.415.442 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=6.) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.455 [api_impl.cc:3100] 619435 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.466 [api_impl.cc:3111] 619435 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.478 [api_impl.cc:3130] 619435 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.490 [p2p_mgmt.cc:171] [619435]local logic id:4, local physic id:4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.503 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.518 [api_impl.cc:3520] 619435 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.533 [p2p_mgmt.cc:178] [619435]disable p2p: local logic id:4, remote physic id:7. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.544 [api_impl.cc:3184] 619435 DisableP2P: Disable P2P devIdDes=4, phyIdSrc=7. [INFO] DRV(619435,python3.9):2025-04-03-09:35:52.415.566 [devdrv_pcie.c:890][ascend][curpid:619435,619435][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=4; peer_phy_devid=7.) [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.581 [hccl_communicator.cc:2043] [619435]Profiling is deinitiated. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.593 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.614 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.415.631 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.648 [opexecounter.cc:103] [619435]free counter mem resource [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.415.662 [atrace_api.c:118](tid:619435) AtraceDestroy start. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.415.678 [tracer_mgr_operate.c:248](tid:619435) destroy object HCCL_619762_4, exitSave(true). [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.415.690 [atrace_api.c:120](tid:619435) AtraceDestroy end. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.415.701 [adapter_trace.cc:33] [619435]Call TraceDestroy, Params:handle[0xaaaaf23809e8] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.416.807 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.416.847 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.416.877 [queue_notify_manager.cc:100] [619435]QueueNotifyManager Destroy. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.416.891 [queue_notify_manager.cc:106] [619435]QueueNotifyManager Destroy success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.021 [queue_notify_manager.cc:100] [619435]QueueNotifyManager Destroy. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.038 [queue_notify_manager.cc:106] [619435]QueueNotifyManager Destroy success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.090 [ccl_buffer_manager.cc:125] [619435][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.108 [ccl_buffer_manager.cc:147] [619435][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.122 [ccl_buffer_manager.cc:160] [619435][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.134 [ccl_buffer_manager.cc:163] [619435][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.147 [ccl_buffer_manager.cc:166] [619435][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.186 [op_base.cc:128] [619435][HcclGetDeviceId] deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.207 [op_base.cc:2196] [619435]op_base comm destroy complete, take time [76902]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[4]. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.417.222 [device.cpp:183]619435 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.238 [api_impl.cc:3419] 619435 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.251 [runtime.cc:3444] 619435 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.263 [api_impl.cc:3449] 619435 DeviceReset: Succ, userDeviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.417.274 [device.cpp:189]619435 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 4. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.417.315 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.417.352 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.417.366 [hcom_executor.cc:40] [619435]Hcom Excutor Finalize start. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.542 [api_impl.cc:3419] 619729 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.547 [api_impl.cc:3419] 619727 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.564 [runtime.cc:3444] 619729 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.575 [runtime.cc:3444] 619727 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.583 [api_impl.cc:3449] 619729 DeviceReset: Succ, userDeviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.590 [api_impl.cc:3449] 619727 DeviceReset: Succ, userDeviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.907 [api_impl.cc:3419] 619730 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.925 [api_impl.cc:3419] 619728 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.934 [runtime.cc:3444] 619730 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.947 [runtime.cc:3444] 619728 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.951 [api_impl.cc:3449] 619730 DeviceReset: Succ, userDeviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.417.962 [api_impl.cc:3449] 619728 DeviceReset: Succ, userDeviceId=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.111 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.418.134 [adapter_rts.cc:321] [619435][hrtCtxSetCurrent] success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.418.148 [stream.cc:182] [619435]Switch Ctx ret[0], curCtx[0xaaab02fde2d0], setCtx[0xaaab02fde2d0], stream id[5] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.166 [api_impl.cc:1228] 619435 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.418.182 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.418.202 [single_op_manager.cc:50]619435 ReleaseResource:ReleaseResource in. resource id = 0xaaab0e381f80 [INFO] GE(619435,python3.9):2025-04-03-09:35:52.418.222 [caching_allocator.cc:402]619435 TryFreeBlocks:Try free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.418.237 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:52.418.272 [npu_memory_allocator.cc:72]619435 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.418.288 [stream_executor.cpp:197]619435 RemoveExecutor: To remove executor by stream = 187651654688640 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.418.334 [acl_resource_manager.cpp:823]619435 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.351 [stream.cc:852] 619435 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.367 [stream.cc:938] 619435 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.381 [stream.cc:1507] 619435 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.408 [stream_sqcq_manage.cc:451] 619435 FreeLogicCq: Return(0), threadIdentifier(281473359609888), devId(4), tsId(0), cqId(323), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.445 [npu_driver.cc:6163] 619435 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=5, logicCq=323. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.418.485 [npu_driver.cc:1180] 619435 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=323, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.420.460 [stream.cc:237] 619435 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.420.491 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.420.506 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.123 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.144 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.162 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.183 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=5, device_id=4, tsId=0, streamFlag=1536 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.421.854 [stream.cc:185] [619435][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab0e381f80], stream id[5] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.873 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.421.888 [adapter_rts.cc:321] [619435][hrtCtxSetCurrent] success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.421.898 [stream.cc:193] [619435]Restore Ctx ret[0], setCtx[0xaaab02fde2d0], stream id[5] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.421.958 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.421.976 [adapter_rts.cc:321] [619435][hrtCtxSetCurrent] success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.421.986 [stream.cc:182] [619435]Switch Ctx ret[0], curCtx[0xaaab02fde2d0], setCtx[0xaaab02fde2d0], stream id[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.000 [api_impl.cc:1228] 619435 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.422.014 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.422.028 [single_op_manager.cc:50]619435 ReleaseResource:ReleaseResource in. resource id = 0xaaab0e305f20 [INFO] GE(619435,python3.9):2025-04-03-09:35:52.422.043 [caching_allocator.cc:402]619435 TryFreeBlocks:Try free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.422.054 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:52.422.063 [npu_memory_allocator.cc:72]619435 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.422.073 [stream_executor.cpp:197]619435 RemoveExecutor: To remove executor by stream = 187651654180640 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.422.085 [acl_resource_manager.cpp:823]619435 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.096 [stream.cc:852] 619435 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.109 [stream.cc:938] 619435 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.119 [stream.cc:1507] 619435 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.150 [stream_sqcq_manage.cc:451] 619435 FreeLogicCq: Return(0), threadIdentifier(281473359609888), devId(4), tsId(0), cqId(322), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.170 [npu_driver.cc:6163] 619435 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=4, logicCq=322. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.422.204 [npu_driver.cc:1180] 619435 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=322, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.424.704 [stream.cc:237] 619435 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.424.722 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.424.732 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.425.201 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.425.217 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.425.229 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.425.245 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=4, device_id=4, tsId=0, streamFlag=1536 [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.425.910 [stream.cc:185] [619435][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab0e305f20], stream id[4] [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.425.930 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.425.945 [adapter_rts.cc:321] [619435][hrtCtxSetCurrent] success [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.425.956 [stream.cc:193] [619435]Restore Ctx ret[0], setCtx[0xaaab02fde2d0], stream id[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.425.987 [hcom_executor.cc:45] [619435][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.426.022 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.426.047 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.061 [hcom_ops_kernel_info_store.cc:3423] [619435]finalize hccl kernel info store. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.077 [hcom_common.cc:85] [619435][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[4] [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.096 [plugin_manager.cc:56] [619435]hccl ops plugin finalize start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.114 [hcom_plugin.cc:90] [619435]Finalize start. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.228 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.244 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.256 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.267 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.289 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.301 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.320 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.329 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.353 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.364 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.373 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.383 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.392 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.401 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.410 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.419 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.429 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.438 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.447 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.457 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.478 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.489 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.498 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.508 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.531 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.542 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.552 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.562 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.586 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.601 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.611 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.622 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.647 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.658 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.667 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.678 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.702 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.713 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.722 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.732 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.758 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.769 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.778 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.787 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.813 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.824 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.832 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.841 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.867 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.878 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.886 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.895 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.922 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.933 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.942 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.951 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.975 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.986 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.426.994 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.004 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.029 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.040 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.048 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.057 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.067 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.076 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.086 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.095 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.106 [hcom_plugin.cc:119] [619435]Finalize: HcomDestroy success. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.117 [plugin_manager.cc:62] [619435]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.427.194 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.427.213 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.231 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.246 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.256 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.267 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.276 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.287 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.295 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.305 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.314 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.326 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.335 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.346 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.356 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.370 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.380 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.391 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.399 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.410 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.422 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.433 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.445 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.458 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.468 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.478 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.489 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.498 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.508 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.519 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.529 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.540 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.548 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.558 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.568 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.580 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.597 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.607 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.616 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.627 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.637 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.647 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.655 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.665 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.674 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.683 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.691 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.702 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.710 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.721 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.729 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.740 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.748 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.757 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.766 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.776 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.785 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.794 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.802 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.813 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.821 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.831 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.839 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.848 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.859 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.867 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.876 [hcom_common.cc:958] [619435][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.889 [hcom_common.cc:961] [619435]Entry-HcomDestroy:void skip [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.898 [hcom.cc:337] [619435][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:52.427.916 [hcom.cc:340] [619435]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.427.939 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.427.957 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.137 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.168 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.189 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.204 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.219 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.232 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.246 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.259 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.273 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.286 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.297 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.309 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.322 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.336 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.350 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.362 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.376 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.411 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.427 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.446 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.463 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.479 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.495 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.509 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.522 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.537 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.552 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.566 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.579 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.595 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.608 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.622 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.643 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffea0100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.659 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffea0101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.428.680 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.699 [callback.cpp:37]619435 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.717 [callback.cpp:44]619435 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.750 [context.cpp:72]619435 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.764 [api_impl.cc:3717] 619435 ContextSetCurrent: set context=aaab02fde2d0, flag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.777 [context.cpp:80]619435 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.791 [stream.cpp:75]619435 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619435,python3.9):2025-04-03-09:35:52.428.809 [stream_manage_func_registry.cc:22]619435 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.821 [stream.cpp:80]619435 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.831 [api_impl.cc:1228] 619435 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.847 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.428.859 [single_op_manager.cc:50]619435 ReleaseResource:ReleaseResource in. resource id = 0xaaab03f47eb0 [INFO] GE(619435,python3.9):2025-04-03-09:35:52.428.875 [caching_allocator.cc:402]619435 TryFreeBlocks:Try free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.428.888 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:52.428.897 [npu_memory_allocator.cc:72]619435 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.907 [stream_executor.cpp:197]619435 RemoveExecutor: To remove executor by stream = 187651482484400 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.428.918 [acl_resource_manager.cpp:823]619435 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.932 [stream.cc:852] 619435 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.948 [stream.cc:938] 619435 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.962 [event.cc:1036] 619435 WaitForBusy: device_id=4, event_id=0 finished map already empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.428.979 [npu_driver.cc:626] 619435 EventIdFree: event id free begin, device_id=4, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.004 [raw_device.cc:1158] 619435 FreeEventIdFromDrv: device_id=4, event_id=0 tear down. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.022 [event.cc:61] 619435 ~Event: device_id=4, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.039 [npu_driver.cc:3803] 619435 SqCqFree: deviceId=4, tsId=0, sqId=84, cqId=84 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.082 [subscribe.cc:244] 619435 Delete: delete callback subscribe: groupId=320, threadId=281471351599008, devId=4, tsId=0, streamId=2, sqId=84, cqId=84 [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.429.087 [trs_sqcq.c:1491][ascend][curpid:619435,619606][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.098 [stream.cc:1507] 619435 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.110 [npu_driver.cc:3833] 619606 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.127 [api_impl.cc:5058] 619606 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.129 [stream_sqcq_manage.cc:451] 619435 FreeLogicCq: Return(0), threadIdentifier(281473359609888), devId(4), tsId(0), cqId(320), isFastCq(0). [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.143 [callback.cpp:132]619606 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.153 [npu_driver.cc:6163] 619435 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=2, logicCq=320. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.159 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.178 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.191 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.200 [npu_driver.cc:1180] 619435 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=320, drvFlag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.206 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.224 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.238 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.252 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.264 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.277 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.292 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.303 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.314 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.328 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.340 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.370 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.382 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.394 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.406 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.420 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.432 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.445 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.459 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.472 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.486 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.496 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.508 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.521 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.533 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.546 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.561 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.573 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.587 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.598 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.612 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.623 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.635 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.647 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.659 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.671 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.690 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.702 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.714 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.725 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.737 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.751 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.764 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.775 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.788 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.800 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.812 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.824 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.835 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.848 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.862 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.874 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.888 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.899 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.910 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.922 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.933 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.947 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.960 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.429.972 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.429.991 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.003 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.016 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.027 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.040 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.054 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.068 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.079 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.092 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.107 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.120 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.134 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.144 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.156 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.169 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.182 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.194 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.205 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.218 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.232 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.245 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.260 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.274 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.286 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.299 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.320 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.333 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.346 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.359 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.371 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.381 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.393 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.408 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.421 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.436 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.447 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.458 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.470 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.483 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.497 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.510 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.521 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.536 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.548 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.561 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.571 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.584 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.597 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.611 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.622 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.640 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.652 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.666 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.676 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.688 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.698 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.709 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.721 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.732 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.744 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.756 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.772 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.785 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.797 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.809 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.819 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.833 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.843 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.856 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.866 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.878 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.890 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.901 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.911 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.923 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.944 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.960 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.430.973 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.986 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.430.998 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.009 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.023 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.036 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.047 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.058 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.070 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.081 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.092 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.105 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.118 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.130 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.142 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.153 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.166 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.180 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.192 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.205 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.217 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.230 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.254 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.268 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.278 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.293 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.304 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.315 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.328 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.339 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.350 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.363 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.375 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.383 [stream.cc:237] 619435 FreeStreamId: Free stream_id=2. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.387 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.402 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.410 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.419 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=4, tsId=0, drvFlag=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.428 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.449 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.461 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.471 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.484 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.495 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.509 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.520 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.531 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.544 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.565 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.578 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.590 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.601 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.612 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.624 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.636 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.647 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.659 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.670 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.684 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.696 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.707 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.720 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.732 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.744 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.760 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.772 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.785 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.797 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.807 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.818 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.830 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.841 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.862 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.875 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.888 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.903 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.903 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=4, tsId=0, drvFlag=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.916 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.928 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.938 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.947 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=2, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.954 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.966 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=2, device_id=4, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.431.972 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.431.993 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.004 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.015 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.027 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.038 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.049 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.061 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.073 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.084 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.097 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.113 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.128 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.144 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.178 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.192 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.204 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.216 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.228 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.240 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.252 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.265 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.276 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.287 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.298 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.310 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.322 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.333 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.344 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.356 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.366 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.378 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.392 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.404 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.416 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.427 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.438 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.451 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.461 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.479 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.493 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.505 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.516 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.530 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.543 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.554 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.568 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.580 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.591 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.604 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.613 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.626 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.637 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.650 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.661 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.673 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.685 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.697 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.708 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.718 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.733 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.746 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.758 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.778 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.789 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.802 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.813 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.823 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.835 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.849 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.859 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.870 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.882 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.896 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.906 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.916 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.928 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.939 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.949 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.961 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.432.972 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.982 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.432.994 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.007 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.018 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.029 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.042 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.053 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.071 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.082 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.094 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.106 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.118 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.130 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.143 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.154 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.168 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.181 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.197 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.212 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.225 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.240 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.253 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.262 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.275 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.288 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.303 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.315 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.327 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.339 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.352 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.363 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.376 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.395 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.406 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.418 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.432 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.446 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.456 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.467 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.479 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.491 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.502 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.512 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.523 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.535 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.549 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.562 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.573 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.585 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.595 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.606 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.619 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.632 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.643 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.655 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.666 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.678 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.701 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.715 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.726 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.740 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.753 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.766 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.778 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.789 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.801 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.812 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.825 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.839 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.849 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.860 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.872 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.883 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.895 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.907 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.918 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.931 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.942 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.953 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.433.964 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.433.978 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.000 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.014 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.026 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.039 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.049 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.060 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.071 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.083 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.097 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.109 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.122 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.133 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.147 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.161 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.174 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.186 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.198 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.213 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.225 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.239 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.250 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.262 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.276 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.288 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.302 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.328 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.343 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.355 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.371 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.386 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.398 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.410 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.421 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.434 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.446 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.457 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.467 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.479 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.491 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.503 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.513 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.527 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.540 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.555 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.567 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.578 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.590 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.603 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.615 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.627 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.646 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.659 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.669 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.681 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.692 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.704 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.716 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.727 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.737 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.748 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.763 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.776 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.788 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.801 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.812 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.822 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.835 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.846 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.857 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.868 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.880 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.894 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.907 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.918 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.929 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.963 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.434.978 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.434.992 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.004 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.018 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.029 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.043 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.058 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.072 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.082 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.094 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.104 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.115 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.125 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.142 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.154 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.165 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.175 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.188 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.201 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.212 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.223 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.234 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.246 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.259 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.279 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.293 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.304 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.317 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.332 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.343 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.345 [stream.cpp:87]619435 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.354 [callback.cpp:116]619606 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.377 [subscribe.cc:308] 619606 GetGroupIdByThreadId: can not find subscribe info, threadId=281471351599008 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.388 [api_impl.cc:5043] 619606 ProcessReport: get groupId fail, threadIdentifier=281471351599008, reCode=0x7160004 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.435.390 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab03f47eb0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.401 [callback.cpp:130]619606 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.418 [stream.cpp:75]619435 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619435,python3.9):2025-04-03-09:35:52.435.436 [stream_manage_func_registry.cc:22]619435 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.447 [stream.cpp:80]619435 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.460 [api_impl.cc:1228] 619435 StreamDestroy: stream_id=3. [INFO] DEVICE(619435,ffff27ed2fa0,python3.9):2025-04-03-09:35:52.435.468 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.478 [acl_resource_manager.cpp:814]619435 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.435.499 [single_op_manager.cc:50]619435 ReleaseResource:ReleaseResource in. resource id = 0xaaab04078dd0 [INFO] GE(619435,python3.9):2025-04-03-09:35:52.435.513 [caching_allocator.cc:402]619435 TryFreeBlocks:Try free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:52.435.525 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:52.435.540 [npu_memory_allocator.cc:72]619435 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.555 [stream_executor.cpp:197]619435 RemoveExecutor: To remove executor by stream = 187651483733456 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.570 [acl_resource_manager.cpp:823]619435 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.584 [stream.cc:852] 619435 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.614 [stream.cc:938] 619435 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.629 [event.cc:1036] 619435 WaitForBusy: device_id=4, event_id=1 finished map already empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.645 [npu_driver.cc:626] 619435 EventIdFree: event id free begin, device_id=4, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.666 [raw_device.cc:1158] 619435 FreeEventIdFromDrv: device_id=4, event_id=1 tear down. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.680 [event.cc:61] 619435 ~Event: device_id=4, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.698 [npu_driver.cc:3803] 619435 SqCqFree: deviceId=4, tsId=0, sqId=85, cqId=85 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.730 [subscribe.cc:244] 619435 Delete: delete callback subscribe: groupId=321, threadId=281471343144864, devId=4, tsId=0, streamId=3, sqId=85, cqId=85 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.747 [stream.cc:1507] 619435 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:52.435.747 [trs_sqcq.c:1491][ascend][curpid:619435,619607][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=4; type=1; ret=16) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.768 [stream_sqcq_manage.cc:451] 619435 FreeLogicCq: Return(0), threadIdentifier(281473359609888), devId(4), tsId(0), cqId(321), isFastCq(0). [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.775 [npu_driver.cc:3833] 619607 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=4, tsId=0, groupId=321, drvRetCode=16. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.789 [npu_driver.cc:6163] 619435 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=3, logicCq=321. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.795 [api_impl.cc:5058] 619607 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.817 [callback.cpp:132]619607 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.834 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.836 [npu_driver.cc:1180] 619435 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=321, drvFlag=0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.853 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.869 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.884 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.897 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.911 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.923 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.937 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.435.950 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.963 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.435.986 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.000 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.015 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.027 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.037 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.050 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.061 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.073 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.083 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.095 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.106 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.120 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.134 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.148 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.172 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.188 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.201 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.217 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.229 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.242 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.251 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.263 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.277 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.292 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.304 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.323 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.336 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.349 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.360 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.370 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.381 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.393 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.408 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.422 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.432 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.444 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.459 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.473 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.488 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.501 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.510 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.521 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.532 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.544 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.560 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.574 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.585 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.598 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.609 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.624 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.651 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.664 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.677 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.690 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.702 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.713 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.725 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.737 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.749 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.763 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.775 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.786 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.797 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.809 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.819 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.832 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.846 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.858 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.867 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.879 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.893 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.908 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.921 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.935 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.955 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.967 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.436.982 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.436.995 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.008 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.019 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.032 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.046 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.056 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.067 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.077 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.090 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.104 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.114 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.124 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.136 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.150 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.163 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.173 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.190 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.203 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.214 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.226 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.239 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.254 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.275 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.285 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.296 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.308 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.320 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.333 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.345 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.356 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.370 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.383 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.394 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.407 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.420 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.432 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.444 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.457 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.470 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.485 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.502 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.516 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.529 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.540 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.553 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.566 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.577 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.598 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.613 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.626 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.639 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.651 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.661 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.674 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.689 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.701 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.712 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.725 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.737 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.748 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.761 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.773 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.786 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.797 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.810 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.823 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.836 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.846 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.857 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.869 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.885 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.898 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.916 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.930 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.941 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.954 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.966 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.437.980 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.437.994 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.005 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.016 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.030 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.044 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.056 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.065 [stream.cc:237] 619435 FreeStreamId: Free stream_id=3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.067 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.085 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.093 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.104 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=4, tsId=0, drvFlag=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.113 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.133 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.143 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.156 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.171 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.185 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.197 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.209 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.231 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.245 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.260 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.273 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.286 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.298 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.312 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.323 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.337 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.351 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.363 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.377 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.391 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.407 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.421 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.434 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.445 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.455 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.466 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.479 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.493 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.505 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.519 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.533 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.545 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.565 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.566 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=4, tsId=0, drvFlag=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.581 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.594 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.602 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.613 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.620 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.634 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=3, device_id=4, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.640 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.662 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.671 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.685 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.699 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.709 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.719 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.732 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.744 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.756 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.768 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.781 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.793 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.805 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.816 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.827 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.849 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.862 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.872 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.885 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.898 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.910 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.920 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.934 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.945 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.957 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.438.969 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.979 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.438.989 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.002 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.014 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.026 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.035 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.046 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.057 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.070 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.082 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.095 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.107 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.120 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.130 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.153 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.166 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.180 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.191 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.203 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.213 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.228 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.240 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.252 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.263 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.276 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.291 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.305 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.317 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.327 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.339 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.351 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.365 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.381 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.394 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.408 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.420 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.432 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.446 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.461 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.478 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.493 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.504 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.516 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.529 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.540 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.551 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.562 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.572 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.585 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.598 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.609 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.621 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.633 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.645 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.659 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.676 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.688 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.701 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.711 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.723 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.735 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.746 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.760 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.782 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.793 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.805 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.820 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.834 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.849 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.862 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.877 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.890 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.904 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.917 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.929 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.945 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.961 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.439.976 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.988 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.439.998 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.009 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.024 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.039 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.051 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.063 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.074 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.089 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.104 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.125 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.138 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.152 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.182 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.198 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.210 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.221 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.236 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.248 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.260 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.273 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.284 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.297 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.308 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.322 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.332 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.347 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.361 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.372 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.382 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.394 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.405 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.418 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.434 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.448 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.471 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.483 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.495 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.508 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.519 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.531 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.547 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.562 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.575 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.589 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.600 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.614 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.624 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.636 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.650 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.662 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.673 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.688 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.699 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.711 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.720 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.733 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.743 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.758 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.771 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.792 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.795 [event.cc:61] 619435 ~Event: device_id=4, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.807 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.829 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.843 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.842 [stream.cpp:87]619435 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.856 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.875 [callback.cpp:116]619607 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.889 [subscribe.cc:308] 619607 GetGroupIdByThreadId: can not find subscribe info, threadId=281471343144864 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.440.888 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab04078dd0. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.902 [api_impl.cc:5043] 619607 ProcessReport: get groupId fail, threadIdentifier=281471343144864, reCode=0x7160004 [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.918 [device.cpp:183]619435 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.922 [callback.cpp:130]619607 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.941 [api_impl.cc:3419] 619435 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.959 [runtime.cc:3444] 619435 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.440.975 [api_impl.cc:3449] 619435 DeviceReset: Succ, userDeviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.440.986 [device.cpp:189]619435 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 4. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.029 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] DEVICE(619435,ffff276c2fa0,python3.9):2025-04-03-09:35:52.441.027 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.077 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.106 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.124 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.441.154 [callback.cpp:37]619435 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:52.441.173 [callback.cpp:44]619435 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.197 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:52.441.217 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.583.520 [awatchdog_monitor.c:160](tid:619468) thread 619727 not exist [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.583.554 [awatchdog_monitor.c:160](tid:619468) thread 619728 not exist [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.583.574 [awatchdog_monitor.c:160](tid:619468) thread 619729 not exist [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:52.583.594 [awatchdog_monitor.c:160](tid:619468) thread 619730 not exist [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.748.719 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.748.748 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.748.762 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:52.748.777 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.748.792 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:52.748.850 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.092.178 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.092.180 [queue_client_comm.c:114][ascend][curpid:619435,619715][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.092.180 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.280 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.294 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.305 [npu_driver.cc:4338] 619715 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.308 [tensor_data_transfer.cpp:1054]619719 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_summary [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.324 [tensor_data_transfer.cpp:612]619715 acltdtReceiveTensorV2: queue is empty, device is 4, name is _npu_log [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.330 [tensor_data_transfer.cpp:1056]619719 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.348 [api_impl.cc:6121] 619719 MemQueuePeek: Start to peek queue on devId 4, qid is 2. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.360 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.373 [npu_driver.cc:4330] 619719 MemQueuePeek: peek mem queue, deviceId=4, qid=2, timeout=800. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.378 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.391 [tensor_data_transfer.cpp:1054]619717 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.404 [tensor_data_transfer.cpp:1056]619717 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.416 [api_impl.cc:6121] 619717 MemQueuePeek: Start to peek queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.428 [npu_driver.cc:4330] 619717 MemQueuePeek: peek mem queue, deviceId=4, qid=1, timeout=800. [INFO] DEVICE(619435,fffefdb36fa0,python3.9):2025-04-03-09:35:53.092.466 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.092.633 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is _npu_log [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.656 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.092.669 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=0. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.093.097 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is _npu_log [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.093.129 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.093.148 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.124.174 [queue_client_comm.c:114][ascend][curpid:619435,619725][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.124.175 [queue_client_comm.c:114][ascend][curpid:619435,619723][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.124.177 [queue_client_comm.c:114][ascend][curpid:619435,619721][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.124.271 [npu_driver.cc:4338] 619725 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.124.289 [tensor_data_transfer.cpp:612]619725 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_histogram_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.124.295 [npu_driver.cc:4338] 619723 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.124.313 [tensor_data_transfer.cpp:612]619723 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_scalar_summary [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.124.321 [npu_driver.cc:4338] 619721 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.124.349 [tensor_data_transfer.cpp:612]619721 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_image_summary [INFO] DEVICE(619435,fffed67cefa0,python3.9):2025-04-03-09:35:53.124.409 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619435,fffed77eefa0,python3.9):2025-04-03-09:35:53.124.423 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619435,fffef4a61fa0,python3.9):2025-04-03-09:35:53.124.471 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.124.648 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is ms_histogram_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.124.671 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.124.684 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=5. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.125.046 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is ms_histogram_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.125.078 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.125.098 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.125.149 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is ms_scalar_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.125.164 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.125.176 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.125.534 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is ms_scalar_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.125.558 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.125.574 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.125.627 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is ms_image_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.125.642 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 3. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.125.653 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=3. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.126.008 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is ms_image_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.126.031 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.126.048 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.924.193 [queue_client_comm.c:114][ascend][curpid:619435,619717][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619435,python3.9):2025-04-03-09:35:53.924.216 [queue_client_comm.c:114][ascend][curpid:619435,619719][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.924.362 [npu_driver.cc:4338] 619717 MemQueuePeek: queue empty [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:53.924.379 [npu_driver.cc:4338] 619719 MemQueuePeek: queue empty [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.924.380 [tensor_data_transfer.cpp:612]619717 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.924.397 [tensor_data_transfer.cpp:612]619719 acltdtReceiveTensorV2: queue is empty, device is 4, name is ms_tensor_summary [INFO] DEVICE(619435,fffef6aa1fa0,python3.9):2025-04-03-09:35:53.924.571 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619435,fffef5a81fa0,python3.9):2025-04-03-09:35:53.924.578 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.924.787 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is ms_tensor_summary [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.924.821 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.924.834 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=2. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.925.229 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is ms_tensor_summary [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.279 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.304 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.925.382 [tensor_data_transfer.cpp:973]619435 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 4, name is ms_tensor_dump [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.925.398 [api_impl.cc:6065] 619435 MemQueueDestroy: Start to destroy queue on devId 4, qid is 1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.925.410 [npu_driver.cc:3989] 619435 MemQueueDestroy: deviceId=4, qid=1. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.925.759 [tensor_data_transfer.cpp:977]619435 acltdtDestroyChannel: acltdtDestroyChannel success, device is 4, name is ms_tensor_dump [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.925.780 [device.cpp:183]619435 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.925.799 [api_impl.cc:3419] 619435 DeviceReset: reset device, device_id=4. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.925.812 [runtime.cc:3444] 619435 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.925.826 [api_impl.cc:3449] 619435 DeviceReset: Succ, userDeviceId=4. [INFO] ASCENDCL(619435,python3.9):2025-04-03-09:35:53.925.836 [device.cpp:189]619435 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 4. [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.859 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.882 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.929 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.945 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.961 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_4 [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.976 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.925.994 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.008 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.034 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.177 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.210 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.226 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.241 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.263 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.288 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.310 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.323 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] PS(619435,ffff3d03afa0,python3.9):2025-04-03-09:35:53.926.343 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] PS(619435,ffff3c82afa0,python3.9):2025-04-03-09:35:53.926.353 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619435,fffef6291fa0,python3.9):2025-04-03-09:35:53.926.353 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(619435,fffeb2fdefa0,python3.9):2025-04-03-09:35:53.926.438 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.539 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.557 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.570 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.588 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.612 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.627 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.640 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.738 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.763 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.798 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.813 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.826 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.842 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.855 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.882 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.896 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.910 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.923 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.973 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.926.987 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.219 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.237 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.262 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.278 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.291 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.305 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:53.927.414 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.964.752 [runtime.cc:5239] 619604 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:53.964.850 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:53.964.865 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619435,python3.9):2025-04-03-09:35:53.964.878 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619604] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.964.898 [runtime.cc:3077] 619604 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=4, devId=4, tdt status=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:53.964.989 [npu_driver.cc:551] 619604 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:54.115.115 [awatchdog_monitor.c:160](tid:619468) thread 619731 not exist [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:54.353.844 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:54.353.987 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.182 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.338 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.365 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.596 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.611 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.678 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.684.691 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.170 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.189 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.200 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.211 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.233 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.244 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.792 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.685.811 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.010 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.025 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.182 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.198 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.486 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.503 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.537 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.575 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.588 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.598 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.609 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.619 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.630 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.640 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.649 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.659 [heartbeat.cc:145] [619435][DeInit]heartbeat deinit begin [INFO] HCCL(619435,python3.9):2025-04-03-09:35:54.686.670 [heartbeat.cc:183] [619435][DeInit]heartbeat deinit end [INFO] DEVICE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:54.686.946 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 4, group name: hccl_world_group [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:54.687.003 [hash_data.cpp:63] >>> (tid:619435) HashData uninitialize success [INFO] PROFILING(619435,python3.9):2025-04-03-09:35:54.687.023 [msprof_reporter_mgr.cpp:270] >>> (tid:619435) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.688.707 [api_impl.cc:2003] 619435 DevFree: device free mem=0x12c041200000 [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:54.689.737 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:54.689.762 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:54.689.915 [adx_dump_record.cpp:229][tid:619435] start to dump uninit [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:54.689.941 [adx_dump_record.cpp:239][tid:619435] dump uninit success [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:54.689.971 [adx_dump_record.cpp:674][tid:619713] exit record file thread [INFO] GE(619435,python3.9):2025-04-03-09:35:54.753.854 [operator_factory_impl.cc:392]619435 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.812 [operator_factory_impl.cc:392]619435 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.857 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.869 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.878 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.889 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.898 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.909 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.918 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.768.928 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.898.316 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.898.491 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] GE(619435,python3.9):2025-04-03-09:35:54.898.562 [opsproto_manager.cc:66]619435 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.958.709 [api_impl.cc:7151] 619435 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.958.915 [api_impl.cc:7135] 619435 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.960.892 [api_impl.cc:7151] 619435 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.961.048 [api_impl.cc:7135] 619435 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.962.492 [api_impl.cc:7121] 619435 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.966.108 [api_impl.cc:7121] 619435 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:54.972.948 [osal.cc:321] 619435 ThreadExit: ThreadExit start. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:55.180.744 [stars_engine.cc:1833] 619604 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:55.180.776 [stars_engine.cc:1836] 619604 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619435,ffff9f9d0020,python3.9):2025-04-03-09:35:55.181.054 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619435,python3.9):2025-04-03-09:35:55.217.417 [ops_kernel_builder_registry.cc:71]619435 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619435,python3.9):2025-04-03-09:35:55.217.480 [ops_kernel_builder_registry.cc:41]619435 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:56.576.226 [adx_server_manager.cpp:236][tid:619714] server manager stop [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:57.073.570 [adx_hdc_epoll.cpp:57][tid:619435] start to close hdc epoll [INFO] GE(619435,python3.9):2025-04-03-09:35:57.154.932 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab08babad0]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.156.881 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab08bb0100]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.159.947 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab08bda580]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.163.465 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab08bad110]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.164.627 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab09fe6700]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.165.723 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab0a00e6f0]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.167.006 [op_impl_registry_holder_manager.cc:31]619435 CloseHandle:start close handle, handle[0xaaab0a0d1d70]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.184.846 [mem_manager.cc:118]619435 Finalize:MemManager Finalize. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.184.885 [session_scope_mem_allocator.cc:73]619435 FreeAllMemory:Free all memory [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.690 [session_scope_mem_allocator.cc:73]619435 FreeAllMemory:Free all memory [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.718 [caching_allocator.cc:474]619435 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.747 [caching_allocator.cc:391]619435 FreeBlocks:Free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.761 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.773 [caching_allocator.cc:422]619435 FreeBlockBins:Free block bins. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.783 [caching_allocator.cc:474]619435 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.813 [caching_allocator.cc:391]619435 FreeBlocks:Free blocks. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.824 [caching_allocator.cc:358]619435 FreeCachedBlocks:Free cached blocks [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.833 [caching_allocator.cc:422]619435 FreeBlockBins:Free block bins. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.849 [mem_manager.cc:145]619435 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.865 [rts_caching_mem_allocator.cc:150]619435 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619435,python3.9):2025-04-03-09:35:57.187.909 [rts_caching_mem_allocator.cc:136]619435 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.268.964 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.024 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.035 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.045 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.055 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.066 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.088 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.100 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.109 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.120 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.128 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.137 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.145 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.152 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.160 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] HCCL(619435,python3.9):2025-04-03-09:35:57.269.169 [network_manager.cc:875] [619435]destroy all nic/vnic. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.363 [runtime.cc:1570] 619435 ~Runtime: deconstruct runtime [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.396 [runtime.cc:1577] 619435 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.827 [raw_device.cc:262] 619435 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.853 [engine.cc:188] 619435 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.876 [ringbuffer_maintain_task.cc:83] 619435 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.892 [stream.cc:1400] 619435 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.906 [stream.cc:2595] 619435 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.931 [npu_driver.cc:751] 619435 SqTaskSend: device_id=4, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.944 [stars_engine.cc:893] 619435 SendTask: Sq task send finished, device_id=4, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.972 [stream.cc:1494] 619435 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.274.988 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.003 [stream.cc:2886] 619435 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.016 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.026 [stream.cc:2886] 619435 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.038 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.050 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.063 [task.cc:185] 619435 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.077 [logger.cc:152] 619435 TaskFinished: device_id=4, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.095 [stars_engine.cc:437] 619435 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.109 [stream.cc:2886] 619435 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.121 [stars_engine.cc:524] 619435 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.133 [npu_driver.cc:5547] 619435 GetSqHead: deviceId=4, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.150 [stream.cc:852] 619435 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.164 [stream.cc:938] 619435 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.275.175 [stream.cc:1507] 619435 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.278.956 [stream_sqcq_manage.cc:451] 619435 FreeLogicCq: Return(0), threadIdentifier(281473359609888), devId(4), tsId(0), cqId(319), isFastCq(0). [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.008 [npu_driver.cc:6163] 619435 StreamUnBindLogicCq: success: deviceId=4, tsId=0, streamId=0, logicCq=319. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.070 [npu_driver.cc:1180] 619435 LogicCqFree: Succ:device_id=4,ts_id=0,type=2,logic_cq_id=319, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.085 [stream.cc:237] 619435 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.098 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.109 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.727 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.744 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.759 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.778 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=0, device_id=4, tsId=0, streamFlag=192 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.279.866 [stars_engine.cc:206] 619605 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.018 [stars_engine.cc:191] 619435 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.032 [osal.cc:225] 619435 Join: mmJoinTask failed, thread=281471352287136 does not exist! [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.044 [stars_engine.cc:150] 619435 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.062 [runtime.cc:5239] 619435 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=4, deviceId=4. [INFO] TDT(619435,python3.9):2025-04-03-09:35:57.280.078 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619435] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.092 [runtime.cc:3003] 619435 StopAicpuExecutor: TsdClose success, userDeviceId=4, devId=4, tdt status=0 [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.280.110 [atrace_api.c:118](tid:619435) AtraceDestroy start. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.280.148 [tracer_mgr_operate.c:248](tid:619435) destroy object RUNTIME_ATRACE_DEV4_TS0, exitSave(false). [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.280.180 [atrace_api.c:120](tid:619435) AtraceDestroy end. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.197 [stream.cc:237] 619435 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.209 [stream_sqcq_manage.cc:216] 619435 DeAllocStreamSqCq: deviceId=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.219 [npu_driver.cc:966] 619435 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.437 [npu_driver.cc:974] 619435 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=4, tsId=0, drvFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.451 [stream_sqcq_manage.cc:228] 619435 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.465 [npu_driver.cc:432] 619435 StreamIdFree: stream id free begin, device_id=4, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.280.479 [npu_driver.cc:441] 619435 StreamIdFree: Stream id free success, stream_id=1, device_id=4, tsId=0, streamFlag=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.289.459 [stars_engine.cc:58] 619435 ~StarsEngine: Destructor. [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.289.492 [engine.cc:84] 619435 ~Engine: Destructor. [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.417 [trs_dev_drv.c:370][ascend][curpid:619435,619435][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=4; fd=37; mmapFd=38) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.442 [devmm_svm.c:2238][ascend][curpid:619435,619435][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=4) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.457 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.476 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.491 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.518 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.531 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev4 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.547 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev4 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.562 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev4 Cached_size:1961984Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.574 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev4 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.590 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev4 Cached_size:2097152Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.602 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev4 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.618 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev4 Cached_size:2097152Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.631 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev4 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.644 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev4 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.659 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev4 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.674 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev4 Cached_size:23068672Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.684 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev4 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.699 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev4 Cached_size:6291456Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.710 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev4 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.725 [svm_mem_statistics.c:140][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev4 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.290.745 [svm_mem_statistics.c:146][ascend][curpid:619435,619435][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev4 Cached_size:16777216Bytes [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.360.622 [drv_log_user.c:623][ascend][curpid:619435,619435][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=4; phyid=4; vfid=0; hostpid=619435; devpid=30682) [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.360.664 [npu_driver.cc:3045] 619435 DeviceClose: Close device success, device_id=4, tsId=0 [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.360.681 [raw_device.cc:125] 619435 ~Device: deconstruct device [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.468.803 [plugin_old_arch.cc:25] 619435 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.468.832 [runtime_keeper.cc:167] 619435 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619435,python3.9):2025-04-03-09:35:57.469.662 [task_fail_callback_data_manager.cc:63] 619435 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.533.365 [stacktrace_signal.c:188](tid:619435) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.581.730 [atrace_client_communication.c:141](tid:619435) send [end] msg. [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.583.249 [hdc_core.c:1982][ascend][curpid:619435,619435][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4283; pid=619435) [INFO] IDEDD(619435,python3.9):2025-04-03-09:35:57.583.279 [hdc_api.cpp:671][tid:619435] connect succ, peer_node: 0, peer_devid: 4 [INFO] DRV(619435,python3.9):2025-04-03-09:35:57.583.613 [hdc_client.c:417][ascend][curpid:619435,619435][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4283) [INFO] ATRACE(619435,python3.9):2025-04-03-09:35:57.583.677 [atrace_client_communication.c:164](tid:619435) send [end] msg success. output_allreduce/worker_1.log0000644000175100017500000672746114773362776017143 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619423,ffffb5717020,python3.9):2025-04-03-09:35:26.746.549 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:26.760.613 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.471 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.573 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.627 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.649 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.669 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.687 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.707 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.724 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.743 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.761 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.777 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.791 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.807 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.823 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.838 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.857 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.871 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.885 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.902 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.916 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.930 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.967 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.177.981 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.001 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.017 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.030 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.058 [ir_data_type_symbol_store.cc:240]619423 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.077 [ir_data_type_symbol_store.cc:240]619423 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.094 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.110 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.124 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.145 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.160 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.174 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.189 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.201 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.216 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.230 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.244 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.257 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.276 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.290 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.303 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.324 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.338 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.353 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.372 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.385 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.399 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.415 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.430 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.443 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.458 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.471 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.486 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.499 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.515 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.527 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.544 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.559 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.572 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.587 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.601 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.615 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.628 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.645 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.659 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.673 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.694 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.706 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.720 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.735 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.748 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.762 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.779 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.794 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.807 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.822 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.835 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.849 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.863 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.877 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.890 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.906 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.921 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.933 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.949 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.962 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.977 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.178.989 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.005 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.020 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.034 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.046 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.069 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.082 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.096 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.107 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.120 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.136 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.149 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.163 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.175 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.188 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.204 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.219 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.232 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.248 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.260 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.274 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.287 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.301 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.314 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.327 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.345 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.360 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.374 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.389 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.400 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.420 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.434 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.450 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.462 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.476 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.489 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.505 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.518 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.532 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.543 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.556 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.573 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.587 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.599 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.613 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.627 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.642 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.657 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.668 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.684 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.697 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.712 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.725 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.739 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.751 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.766 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.787 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.800 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.814 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.827 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.841 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.855 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.869 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.881 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.896 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.909 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.926 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.941 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.959 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.979 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.179.994 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.008 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.026 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.040 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.052 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.068 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.080 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.096 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.110 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.124 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.141 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.153 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.177 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.192 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.209 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.225 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.240 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.256 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.268 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.284 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.298 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.314 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.329 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.345 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.374 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.389 [ir_data_type_symbol_store.cc:173]619423 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.401 [ir_data_type_symbol_store.cc:219]619423 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619423,python3.9):2025-04-03-09:35:27.180.420 [ir_data_type_symbol_store.cc:182]619423 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.263 [awatchdog_core.c:99](tid:619423) watchdog get platform host side [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.427 [awatchdog_monitor.c:127](tid:619423) create monitor thread successfully, tid :281473088626592 [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.450 [awatchdog_monitor.c:226](tid:619423) AwdMonitorInit [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.633 [trace_attr.c:99](tid:619423) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.661 [trace_attr.c:189](tid:619423) attr init success, timeout=0ms. [WARNING] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.675 [trace_system_api.c:115](tid:619423) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.734 [trace_recorder.c:40](tid:619423) home_directory=/home/jenkins. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.751 [trace_recorder.c:121](tid:619423) use root path: /home/jenkins/ascend [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.764 [trace_event.c:151](tid:619423) init trace event success [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.187.819 [atrace_client_core.c:185](tid:619423) atrace client init success. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.164 [stacktrace_dumper.c:218](tid:619423) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.189 [trace_system_api.c:115](tid:619423) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.268 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 2 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.285 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 15 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.296 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 3 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.309 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 4 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.321 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 5 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.335 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 6 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.345 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 7 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.357 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 8 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.370 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 11 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.382 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 24 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.394 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 25 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.408 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 31 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.419 [stacktrace_signal.c:165](tid:619423) register signal handler for signo 35 succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.430 [stacktrace_signal.c:168](tid:619423) trace signal init succeed. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:27.189.441 [trace_core.c:51](tid:619423) trace init success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.190.141 [task_fail_callback_data_manager.cc:58] 619423 TaskFailCallBackManager: Constructor. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.200.034 [operator_factory_impl.cc:232]619423 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619423,python3.9):2025-04-03-09:35:27.200.067 [operator_factory_impl.cc:246]619423 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619423,python3.9):2025-04-03-09:35:27.200.079 [operator_factory_impl.cc:239]619423 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619423,python3.9):2025-04-03-09:35:27.207.366 [adapter_rts.cc:2720] [619423][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.223.303 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.223.378 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:27.223.410 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:27.234.419 [prof_cann_plugin.cpp:256] (tid:619423) Register module[6] callback. [INFO] OP(619423,python3.9):2025-04-03-09:35:27.284.790 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619423] cachelimit is 10000 [INFO] OP(619423,python3.9):2025-04-03-09:35:27.288.639 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619423] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619423,python3.9):2025-04-03-09:35:27.288.690 [op_dfx.cpp:54][NNOP][SystemConfig][619423] enable debug kernel flag 0. [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:27.288.709 [prof_cann_plugin.cpp:256] (tid:619423) Register module[61] callback. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.343.813 [mem_manager.cc:138]619423 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.351.082 [cache_policy.cc:47]619423 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.059 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.195 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.301 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.329 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.351 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.371 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.392 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.412 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.435 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.456 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.476 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.495 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.515 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.535 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.555 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.574 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.592 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.616 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.636 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.655 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.762 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.881 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.366.905 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.001 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.024 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.049 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.069 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.090 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.112 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.133 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.153 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.172 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.193 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.212 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.232 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.253 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.273 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.293 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.313 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.336 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.360 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.380 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.484 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.589 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.688 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.792 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.896 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.367.920 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.018 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.122 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.146 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.248 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.366 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.390 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.412 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.432 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.455 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.476 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.498 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.518 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.539 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.562 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.582 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.606 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.630 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.651 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.743 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.843 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.368.947 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.056 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.080 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.105 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.127 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.227 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.326 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.350 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.447 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.474 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.567 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.677 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.775 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.369.873 [addn.cc:29]619423 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.228 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.327 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.421 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.514 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.611 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.708 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.798 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.891 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.907 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619423,python3.9):2025-04-03-09:35:27.370.998 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619423,python3.9):2025-04-03-09:35:27.371.091 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619423,python3.9):2025-04-03-09:35:27.371.195 [symbolic_kernel_factory.cc:27]619423 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619423,python3.9):2025-04-03-09:35:27.377.974 [process_node_engine_manager.cc:140]619423 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619423,python3.9):2025-04-03-09:35:27.378.108 [process_node_engine_manager.cc:140]619423 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619423,python3.9):2025-04-03-09:35:27.384.836 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619423,python3.9):2025-04-03-09:35:27.384.871 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619423,python3.9):2025-04-03-09:35:27.385.365 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619423] start packet count timer [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:27.406.462 [prof_atls_plugin.cpp:117] (tid:619423) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:27.406.506 [prof_atls_plugin.cpp:117] (tid:619423) RegisterProfileCallback, callback type is 8 [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.482.683 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.483.182 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.530.524 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.530.661 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 1 [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.530.705 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.531.504 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.171 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.250 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.289 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.384 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.415 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.472 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.533.499 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.535.254 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.535.287 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.535.325 [acl.cpp:753]619423 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.535.399 [runtime_keeper.cc:78] 619423 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.263 [plugin_old_arch.cc:16] 619423 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaaeb678ba0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.297 [runtime_keeper.cc:149] 619423 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.308 [runtime_keeper.cc:221] 619423 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.335 [runtime.cc:1387] 619423 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.367 [runtime.cc:1163] 619423 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.392 [npu_driver.cc:4062] 619423 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.413 [runtime.cc:5096] 619423 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.428 [runtime.cc:1173] 619423 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.440 [runtime.cc:1183] 619423 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.458 [runtime.cc:1193] 619423 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.473 [profiler.cc:84] 619423 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.494 [runtime.cc:1209] 619423 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.526 [runtime.cc:1219] 619423 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.539 [runtime.cc:1230] 619423 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.550 [runtime.cc:1242] 619423 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.563 [runtime.cc:1264] 619423 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.575 [runtime.cc:1270] 619423 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.539.690 [runtime.cc:109] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.089 [runtime.cc:121] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.179 [runtime.cc:1314] 619423 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.199 [runtime.cc:1322] 619423 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.322 [runtime.cc:1338] 619423 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.392 [utils.cc:105] 619423 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.444 [runtime.cc:1664] 619423 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.484 [runtime.cc:1688] 619423 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.505 [runtime.cc:1953] 619423 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.535 [runtime.cc:1477] 619423 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:27.540.556 [prof_atls_plugin.cpp:210] (tid:619423) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.576 [heterogenous.cc:19] 619423 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:27.540.609 [api_c_soc.cc:70] 619423 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.540.625 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.540.640 [acl.cpp:760]619423 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.540.675 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.578.019 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.611.995 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.612.468 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.612.917 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.613.728 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.614.176 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.615.909 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.615.953 [acl.cpp:753]619423 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.615.979 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:27.615.992 [acl.cpp:760]619423 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:27.616.018 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.529.875 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.531.856 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.553.354 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.562.260 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.565.261 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.568.298 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.596.573 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619423,python3.9):2025-04-03-09:35:28.612.629 [platform_info.cpp:853]619423 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.594.661 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.596.446 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.596.582 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.598.837 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.598.973 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.599.090 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.599.217 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.599.323 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.600.451 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.600.588 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.602.511 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.602.642 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.602.756 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.602.867 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.602.970 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619423,python3.9):2025-04-03-09:35:29.603.074 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619423,python3.9):2025-04-03-09:35:30.255.921 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619423,python3.9):2025-04-03-09:35:34.495.857 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.761.709 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.761.802 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.761.837 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619423,python3.9):2025-04-03-09:35:34.807.911 [ops_kernel_builder_registry.cc:55]619423 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619423,python3.9):2025-04-03-09:35:34.808.029 [ops_kernel_builder_registry.cc:28]619423 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.847.067 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.847.179 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.847.304 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.847.325 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:34.847.339 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619423,python3.9):2025-04-03-09:35:35.682.445 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:37.466.543 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:37.467.281 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:37.479.441 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.579.357 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:37.579.470 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.580.937 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.580.991 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.012 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.042 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.320 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281472113909664 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.462 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281472105455520 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.485 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.705 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281472023916448 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.843 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281472015462304 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.870 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.581.887 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.582.126 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58712, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:37.582.166 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [WARNING] DISTRIBUTED(619423,ffff4fffefa0,python3.9):2025-04-03-09:35:37.582.185 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58712 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.279 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.318 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.331 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.346 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.456 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58744, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.082.476 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619423,ffff555d1fa0,python3.9):2025-04-03-09:35:38.082.501 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58744 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.582.555 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.582.579 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.582.912 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 1 has been registered successfully. [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:38.583.063 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619423,ffff4efdefa0,python3.9):2025-04-03-09:35:38.583.115 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619423,ffff4efdefa0,python3.9):2025-04-03-09:35:38.583.139 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:39.083.142 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:39.583.237 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.083.314 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(4/1200). [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.408 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3002.41 msec. [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.431 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.445 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.460 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 1 rank id: 1 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.476 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.544 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 8374 to 8629 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.565 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.117 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.595 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.616 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3002.74 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.637 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.756 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.773 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.798 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.826 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.841 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.583.855 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.585.551 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.585.572 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 8374 to 8629 [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.585.758 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:354f174a-0c38-4e84-8cfe-131801820c34, the ip:127.0.0.1, the port:8374 [INFO] PS(619423,ffff4e7cefa0,python3.9):2025-04-03-09:35:40.585.778 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619423,ffff4e7cefa0,python3.9):2025-04-03-09:35:40.585.801 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.587.386 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.587.528 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_1 has been registered successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.587.550 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:40.587.663 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.316 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 1 has been started successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.356 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.402 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.418 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.446 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 1, global rank size: 8 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.471 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3004.81 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.489 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 1 is devserver-c233-2 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.593 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.638 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.657 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 1 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.673 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 1 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.697 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [1]. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.727 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.235 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.794 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.821 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.588.841 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.588.871 [device.cpp:365]619423 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.588.989 [api_impl.cc:249] 619423 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.589.005 [api_impl.cc:6795] 619423 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.589.018 [device.cpp:371]619423 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.589.080 [callback.cpp:37]619423 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.589.108 [callback.cpp:44]619423 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.589.131 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.589.152 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.589.187 [device.cpp:143]619423 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.589.279 [npu_driver.cc:124] 619423 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.589.301 [api_impl.cc:2999] 619423 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.589.333 [runtime.cc:3602] 619423 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.589.637 [hdc_core.c:1982][ascend][curpid:619423,619423][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4230; pid=619423) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.589.866 [runtime.cc:3609] 619423 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.589.932 [atrace_client_communication.c:103](tid:619423) send [hello] msg. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.589.955 [atrace_client_communication.c:63](tid:619423) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.590.082 [hdc_core.c:1982][ascend][curpid:619423,619423][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4231; pid=619423) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:43.590.102 [hdc_api.cpp:671][tid:619423] connect succ, peer_node: 0, peer_devid: 1 [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.591.463 [hdc_client.c:417][ascend][curpid:619423,619423][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4231) [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.591.504 [atrace_client_communication.c:65](tid:619423) send msg by adx[finish], ret=0. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.591.521 [atrace_client_communication.c:97](tid:619423) msgType=1, magic=49959, version=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.591.556 [atrace_client_communication.c:135](tid:619423) send [hello] msg success. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.591.618 [atrace_client_core.c:160](tid:619423) start to receive device[1] trace log. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.591.658 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.591.679 [runtime.cc:2855] 619423 startAicpuExecutor: TsdOpen devId=1, device mode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.591.695 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.591.738 [hdc_core.c:1982][ascend][curpid:619423,619575][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4232; pid=619423) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:43.591.766 [hdc_api.cpp:671][tid:619575] connect succ, peer_node: 0, peer_devid: 1 [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.773 [client_manager.cpp:160][GetPlatformInfo][tid:619423] [TsdClient] mode[1] chipType[5] [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.591.812 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619423] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.829 [client_manager.cpp:582][GetVisibleDevices][tid:619423] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.844 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.858 [client_manager.cpp:125][GetInstance][tid:619423] [ClientManager] GetInstance, deviceId[1], g_runningMode[0], begin saving instance [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.902 [client_manager.cpp:435][IsHostEnvironment][tid:619423] [TsdClient] logicDeviceId is [1], hostaicpunum[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.921 [client_manager.cpp:464][GetClientRunMode][tid:619423] runningMode:0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.934 [client_manager.cpp:127][GetInstance][tid:619423] [ClientManager] Current mode:1 [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.591.947 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619423] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.591.961 [client_manager.cpp:347][GetProfilingMode][tid:619423] Get AICPU_PROFILING_MODE[] [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.591.988 [atrace_client_communication.c:177](tid:619575) send [start] msg. [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.591.995 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619423] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.592.010 [process_mode_base.cpp:149][GetLogLevel][tid:619423] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.592.039 [atrace_client_communication.c:189](tid:619575) send [start] msg success. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.592.144 [process_mode_base.cpp:161][GetLogLevel][tid:619423] [TsdClient] Get Log Level[001] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.046 [process_mode_manager.cpp:71][OpenProcess][tid:619423] [ProcessModeManager] enter into open process deviceId[1] rankSize[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.072 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619423] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.156 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[.], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.295 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[scene.info], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.457 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.604 [client_manager.cpp:264][CheckPackageExists][tid:619423] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.623 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[ascend_install.info], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.733 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[version.info], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.865 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[..], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.594.998 [client_manager.cpp:261][CheckPackageExists][tid:619423] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.273 [client_manager.cpp:268][CheckPackageExists][tid:619423] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.293 [client_manager.cpp:283][CheckPackageExists][tid:619423] [TsdClient][deviceId=1] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.305 [client_manager.cpp:286][CheckPackageExists][tid:619423] [TsdClient][deviceId=1] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.595.372 [devdrv_hccl.c:103][ascend][curpid:619423,619423][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619423) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.392 [process_mode_manager.cpp:396][InitTsdClient][tid:619423] [TsdClient][deviceId=1] get process sign success, procPid[619423] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.405 [process_mode_manager.cpp:403][InitTsdClient][tid:619423] [TsdClient] deviceId[1] begin to init hdc client [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.441 [hdc_common.cpp:67][InitMsgSize][tid:619423] HdcCommon::InitMsgSize Start [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.458 [hdc_common.cpp:80][InitMsgSize][tid:619423] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.474 [hdc_client.cpp:184][CreateHdcSession][tid:619423] HdcClient::CreateHdcSession Start [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.573 [hdc_client.cpp:208][CreateHdcSession][tid:619423] [HdcClient] deviceId: 1 connect session [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.595.606 [hdc_core.c:1982][ascend][curpid:619423,619423][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4235; pid=619423) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.625 [hdc_client.cpp:219][CreateHdcSession][tid:619423] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.640 [hdc_client.cpp:230][CreateHdcSession][tid:619423] [HdcClient] deviceId: 1 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.657 [version_verify.cpp:36][SetVersionInfo][tid:619423] VersionVerify: send client version to server [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.706 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.731 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.595.756 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.595.792 [internal_api.cpp:101][GetScheduleEnv][tid:619423] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.607 [version_verify.cpp:68][PeerVersionCheck][tid:619423] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.632 [version_verify.cpp:89][ParseVersionInfo][tid:619423] VersionVerify: pass client version info success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.648 [hdc_client.cpp:275][CheckHdcConnection][tid:619423] Service[2] create hdc success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.667 [version_verify.cpp:122][SpecialFeatureCheck][tid:619423] VersionVerify: new type[35], supported [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.597.683 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619423] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.715 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619423] [TsdClient][deviceId=1] [sessionId=1] wait package info respond [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.597.734 [hdc_client.cpp:154][TsdRecvData][tid:619423] recv timeout is:150000 ms [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.754 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619423] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.773 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619423] [PackageInfoMsgProc] [sessionID=1] deviceId[1], checkCode[56684994] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.788 [hdc_client.cpp:172][TsdRecvData][tid:619423] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.802 [version_verify.cpp:122][SpecialFeatureCheck][tid:619423] VersionVerify: new type[64], supported [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.816 [hdc_client.cpp:411][Destroy][tid:619423] enter HdcClient::Destroy() function [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.598.833 [hdc_client.c:417][ascend][curpid:619423,619423][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4235) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.857 [hdc_client.cpp:394][DestroyClient][tid:619423] begin drvHdcClientDestroy [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.875 [hdc_client.cpp:399][DestroyClient][tid:619423] end drvHdcClientDestroy [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.887 [hdc_client.cpp:377][ClearClientPtr][tid:619423] begin HdcClient::ClearClientPtr [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.900 [hdc_client.cpp:385][ClearClientPtr][tid:619423] end HdcClient::ClearClientPtr [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.912 [hdc_client.cpp:418][Destroy][tid:619423] end HdcClient::Destroy() function [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.954 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619423] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.598.974 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619423] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.598.992 [devdrv_hccl.c:103][ascend][curpid:619423,619423][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619423) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.009 [process_mode_manager.cpp:396][InitTsdClient][tid:619423] [TsdClient][deviceId=1] get process sign success, procPid[619423] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.022 [process_mode_manager.cpp:403][InitTsdClient][tid:619423] [TsdClient] deviceId[1] begin to init hdc client [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.039 [hdc_common.cpp:67][InitMsgSize][tid:619423] HdcCommon::InitMsgSize Start [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.062 [hdc_common.cpp:80][InitMsgSize][tid:619423] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.076 [hdc_client.cpp:184][CreateHdcSession][tid:619423] HdcClient::CreateHdcSession Start [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.161 [hdc_client.cpp:208][CreateHdcSession][tid:619423] [HdcClient] deviceId: 1 connect session [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.599.191 [hdc_core.c:1982][ascend][curpid:619423,619423][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4238; pid=619423) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.210 [hdc_client.cpp:219][CreateHdcSession][tid:619423] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.223 [hdc_client.cpp:230][CreateHdcSession][tid:619423] [HdcClient] deviceId: 1 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.238 [version_verify.cpp:36][SetVersionInfo][tid:619423] VersionVerify: send client version to server [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.252 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.268 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.599.281 [version_verify.cpp:52][SetVersionInfo][tid:619423] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.522 [version_verify.cpp:68][PeerVersionCheck][tid:619423] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.544 [version_verify.cpp:89][ParseVersionInfo][tid:619423] VersionVerify: pass client version info success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.562 [hdc_client.cpp:275][CheckHdcConnection][tid:619423] Service[2] create hdc success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.579 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619423] [TsdClient] tsd get process sign successfully, procpid[619423] signSize[48] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.598 [version_verify.cpp:114][SpecialFeatureCheck][tid:619423] VersionVerify: previous type[6], supported [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.624 [process_mode_manager.cpp:88][OpenProcess][tid:619423] [ProcessModeManager] deviceId[1] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619423,python3.9):2025-04-03-09:35:43.601.643 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619423] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.601.655 [hdc_client.cpp:154][TsdRecvData][tid:619423] recv timeout is:150000 ms [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.804.319 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619423] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.804.345 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619423] [TsdClient] DeviceMsgProc recvMsg realDeviceId[1] msgType[8] localDevId[1] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.804.358 [hdc_client.cpp:172][TsdRecvData][tid:619423] [deviceId=1] the Tsd recv data already [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.804.375 [devmm_svm.c:132][ascend][curpid:619423,619423][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.806.611 [devmm_virt_interface.c:450][ascend][curpid:619423,619423][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.807.041 [drv_log_user.c:623][ascend][curpid:619423,619423][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=1; devid=1; vfid=0; hostpid=619423; devpid=31531) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.807.082 [devmm_svm.c:122][ascend][curpid:619423,619423][drv][devmm][devmm_set_device_info]Device info. (devid=1; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.807.365 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619423] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.807.388 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619423] enter into OpenInHost deviceid[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.807.418 [client_manager.cpp:435][IsHostEnvironment][tid:619423] [TsdClient] logicDeviceId is [1], hostaicpunum[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.807.437 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619423] host cpu not support [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.807.449 [process_mode_manager.cpp:121][OpenProcess][tid:619423] [TsdClient][deviceId=1] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.807.467 [runtime.cc:2879] 619423 startAicpuExecutor: userDeviceId=1, devId=1, tsdOpen ret=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.807.485 [runtime.cc:2891] 619423 startAicpuExecutor: TsdOpen success, devId=1, ret=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.807.503 [runtime.cc:3627] 619423 DeviceRetain: new dev ok id=1, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.807.518 [raw_device.cc:936] 619423 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.807.539 [npu_driver.cc:2968] 619423 DeviceOpen: Open device start, device_id=1, tsId=0. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.807.629 [trs_sqcq.c:458][ascend][curpid:619423,619423][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.808.131 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=6; num=2044) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.812.106 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=7; num=2044) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.812.394 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=10; num=1024) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.812.643 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=11; num=1024) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.813.125 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=12; num=2048) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.813.155 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=8; num=16) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.813.180 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=9; num=16) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.813.202 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=13; num=4) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.813.223 [trs_sqcq.c:394][ascend][curpid:619423,619423][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=1; tsId=0; id_type=14; num=10) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.815.524 [trs_dev_drv.c:302][ascend][curpid:619423,619423][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=1; fd=36; mmapFd=37) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.815.563 [devmm_svm.c:132][ascend][curpid:619423,619423][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.815.601 [devmm_svm.c:122][ascend][curpid:619423,619423][drv][devmm][devmm_set_device_info]Device info. (devid=1; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.815.723 [npu_driver.cc:3011] 619423 DeviceOpen: Open device success, device_id=1, tsId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.815.747 [raw_device.cc:292] 619423 Init: chipType:0x5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.815.792 [raw_device.cc:304] 619423 Init: isAddrFlat:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.815.813 [raw_device.cc:320] 619423 Init: GetDevInfo, device_id=1, hardwareVersion=0x503 [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.815.830 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.815.845 [npu_driver.cc:1578] 619423 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.816.844 [raw_device.cc:352] 619423 Init: Get phy info, phy chip id 1, phy die id 0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.816.870 [runtime.cc:2771] 619423 GetTsdQos: [TSD] start get qos, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.816.884 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.898 [tsd_client.cpp:189][TsdCapabilityGet][tid:619423] TsdCapabilityGet Begin. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.910 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.924 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.940 [process_mode_manager.cpp:1042][CapabilityGet][tid:619423] [ProcessModeManager] enter into CapabilityGet process deviceId[1] type[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.958 [process_mode_manager.cpp:386][InitTsdClient][tid:619423] [TsdClient] tsd client has been inited before [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.816.979 [version_verify.cpp:114][SpecialFeatureCheck][tid:619423] VersionVerify: previous type[38], supported [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.026 [process_mode_manager.cpp:1065][CapabilityGet][tid:619423] [TsdClient][deviceId=1] [sessionId=1] [type=0]send capability success [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.045 [hdc_client.cpp:154][TsdRecvData][tid:619423] recv timeout is:150000 ms [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.542 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619423] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.585 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619423] [TsdClient] PidQosMsgProc recvMsg realDeviceId[1] msgType[39] localDevId[1] rspCode[0],pidqos[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.603 [hdc_client.cpp:172][TsdRecvData][tid:619423] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.817.619 [process_mode_manager.cpp:1070][CapabilityGet][tid:619423] [TsdClient][logicDeviceId_=1][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.640 [runtime.cc:2782] 619423 GetTsdQos: [TSD] get qos success, userDeviceId=1, devId=1, qos=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.704 [engine.cc:78] 619423 Engine: Constructor. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.720 [stars_engine.cc:51] 619423 StarsEngine: Constructor. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.731 [engine_factory.cc:26] 619423 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.746 [engine_factory.cc:38] 619423 CreateEngine: Engine init success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.759 [raw_device.cc:364] 619423 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.773 [spm_pool.cc:42] 619423 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.817.989 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.818.009 [spm_pool.cc:55] 619423 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.818.042 [raw_device.cc:380] 619423 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.818.061 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.818.076 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.818.088 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.818.102 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.825.416 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.825.459 [task_allocator.cc:46] 619423 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.825.490 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.825.501 [h2d_copy_mgr.cc:45] 619423 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.825.513 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.825.525 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.826.588 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.826.609 [task_allocator.cc:46] 619423 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.826.624 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.826.635 [h2d_copy_mgr.cc:45] 619423 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.826.650 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.826.663 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.464 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.507 [task_allocator.cc:46] 619423 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.537 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.549 [h2d_copy_mgr.cc:45] 619423 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.559 [uma_arg_loader.cc:86] 619423 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.570 [task_allocator.cc:46] 619423 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.594 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.827.607 [uma_arg_loader.cc:93] 619423 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.435 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.455 [uma_arg_loader.cc:102] 619423 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.467 [uma_arg_loader.cc:104] 619423 Init: ALLOC PCIE is support[1] [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.828.480 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.493 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.620 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.636 [h2d_copy_mgr.cc:45] 619423 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.694 [raw_device.cc:393] 619423 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.720 [raw_device.cc:398] 619423 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.739 [task_allocator.cc:46] 619423 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.828.917 [buffer_allocator.cc:61] 619423 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.159 [raw_device.cc:109] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.553 [raw_device.cc:121] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.572 [raw_device.cc:425] 619423 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.631 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.719 [stream.cc:2246] 619423 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.769 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.829.789 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.382 [npu_driver.cc:6125] 619423 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.439 [npu_driver.cc:6137] 619423 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=409, streamId=0, logicCq=409, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.463 [npu_driver.cc:6189] 619423 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=0, logicCq=409. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.495 [npu_driver.cc:5916] 619423 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d50c5000, addr=dfffd50c5000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.512 [stream.cc:653] 619423 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.561 [stream.cc:276] 619423 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.579 [stream.cc:688] 619423 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=1, streamResId=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.601 [stream_state_callback_manager.cc:45] 619423 Notify: not reg stream state callback func! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.616 [raw_device.cc:593] 619423 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.635 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.655 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=16 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.670 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.758 [tsch_stream.cc:45] 619423 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=1 streamResId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.776 [raw_device.cc:1219] 619423 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.830.910 [osal.cc:309] 619423 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.082 [osal.cc:182] 619423 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.104 [osal.cc:309] 619423 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.183 [osal.cc:182] 619423 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.181 [stars_engine.cc:1785] 619592 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.203 [stars_engine.cc:201] 619593 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.237 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.831.258 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.831.278 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.831.295 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.321 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.367 [npu_driver.cc:6306] 619592 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.390 [stars_engine.cc:1855] 619592 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.406 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.518 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.554 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.595 [task_info.cc:1596] 619423 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.614 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.628 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.661 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.679 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.700 [stars_engine.cc:1122] 619423 SyncTask: Task Wait: device_id=1, stream_id=0, task_id=0, logicCqId=409. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.735 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.751 [stars_engine.cc:1071] 619423 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.777 [stars_engine.cc:1017] 619423 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.797 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.809 [stars_engine.cc:1027] 619423 ProcReport: taskType=97. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.834 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.849 [stars_engine.cc:1570] 619423 ProcLogicCqReport: CQE warning, device_id=1, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.868 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.885 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.902 [task_info.cc:1614] 619423 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.916 [raw_device.cc:1305] 619423 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.952 [stars_engine.cc:1351] 619423 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.973 [npu_driver.cc:5618] 619423 GetSqEnable: deviceId=1, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.831.988 [stars_engine.cc:1365] 619423 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.002 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.018 [npu_driver.cc:5595] 619423 GetSqTail: deviceId=1, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.034 [npu_driver.cc:5406] 619423 SetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.051 [npu_driver.cc:5357] 619423 EnableSq: deviceId=1, tsId=0, sqId=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.067 [stars_engine.cc:1430] 619423 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.093 [npu_driver.cc:5618] 619423 GetSqEnable: deviceId=1, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.108 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.123 [raw_device.cc:1331] 619423 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.832.137 [raw_device.cc:610] 619423 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.421 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.507 [ringbuffer_maintain_task.cc:98] 619423 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x1740 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.524 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.539 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.560 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.575 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.590 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.606 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.650 [stars_engine.cc:622] 619423 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=409, retCode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.667 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.678 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.690 [stream.cc:2886] 619423 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.700 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.712 [stream.cc:2886] 619423 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.723 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.734 [stream.cc:2886] 619423 StarsWaitForTask: No.3 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.747 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.758 [stream.cc:2886] 619423 StarsWaitForTask: No.4 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.768 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.779 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.793 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.805 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.838 [stars_engine.cc:437] 619423 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.852 [stream.cc:2886] 619423 StarsWaitForTask: No.5 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.909 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.927 [task_info.cc:111] 619423 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.942 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.953 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.968 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.979 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.833.993 [stars_engine.cc:1122] 619423 SyncTask: Task Wait: device_id=1, stream_id=0, task_id=2, logicCqId=409. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.128 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.145 [stars_engine.cc:1071] 619423 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.158 [stars_engine.cc:1017] 619423 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.174 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.184 [stars_engine.cc:1027] 619423 ProcReport: taskType=29. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.193 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.203 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.214 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.236 [npu_driver.cc:5618] 619423 GetSqEnable: deviceId=1, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.247 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.834.257 [raw_device.cc:1232] 619423 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.834.280 [atrace_api.c:40](tid:619423) AtraceCreate start. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.426 [trace_rb_log.c:45](tid:619423) init ring buffer time finished, realTime 1743644143834359721, monotonicTime 4211304430183, freq 99995 kHz, minutesWest 480 minutes [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.444 [trace_rb_log.c:137](tid:619423) [RUNTIME_ATRACE_DEV1_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.458 [tracer_mgr_operate.c:66](tid:619423) create object RUNTIME_ATRACE_DEV1_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.478 [atrace_api.c:44](tid:619423) AtraceCreate end. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.530 [trace_event.c:70](tid:619423) limitedNum 1, processType 0 [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.835.544 [trace_event.c:372](tid:619423) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.561 [runtime.cc:3159] 619423 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619423,python3.9):2025-04-03-09:35:43.835.576 [client_manager.cpp:196][SetProfilingCallback][tid:619423] [TsdClient] set profiling callback success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.625 [runtime.cc:3364] 619423 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.659 [context.cc:109] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.772 [context.cc:121] 619423 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.787 [context.cc:276] 619423 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.836 [raw_device.cc:803] 619423 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.850 [npu_driver_dcache_lock.cpp:39] 619423 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.835.966 [raw_device.cc:528] 619423 AllocStackPhyBase: AllocStackPhyBase device_id=1, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.013 [raw_device.cc:232] 619423 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.035 [runtime.cc:3393] 619423 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.051 [runtime.cc:3560] 619423 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.069 [api_impl.cc:2987] 619423 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.083 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.096 [api_impl.cc:2992] 619423 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:43.836.115 [awatchdog_core.c:175](tid:619423) create watchdog for id : 65543, tid : 619423, timeout : 300s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.836.128 [api_impl.cc:3025] 619423 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.836.176 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.836.191 [acl.cpp:121]619423 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.475 [api_impl.cc:3476] 619423 GetDeviceInfo: get device info, deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.511 [api_impl.cc:3476] 619423 GetDeviceInfo: get device info, deviceId=1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.839.554 [device.cpp:133]619423 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 1, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.839.575 [device.cpp:154]619423 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.839.592 [context.cpp:87]619423 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.839.620 [context.cpp:98]619423 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.839.641 [stream.cpp:48]619423 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.667 [api_impl.cc:1120] 619423 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.738 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.776 [stream.cc:2246] 619423 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.815 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.839.830 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.243 [npu_driver.cc:6125] 619423 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.296 [npu_driver.cc:6137] 619423 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=410, streamId=2, logicCq=410, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.320 [npu_driver.cc:6189] 619423 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=2, logicCq=410. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.335 [npu_driver.cc:5916] 619423 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc5e3000, addr=dfffcc5e3000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.349 [stream.cc:653] 619423 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.840.399 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.417 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.840.586 [stream.cc:291] 619423 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.545 [stream.cc:688] 619423 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=1, streamResId=65535 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.581 [stream_state_callback_manager.cc:45] 619423 Notify: not reg stream state callback func! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.596 [api_impl.cc:1162] 619423 StreamCreate: Succ, flags=1536, stream_id=2, context=aaaaf7bf5f00 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.842.620 [stream.cpp:189]619423 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.660 [stream.cc:3161] 619423 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.842.671 [stream.cpp:195]619423 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.842.730 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaaf8b5f1c0. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.842.761 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.842.775 [stream.cpp:48]619423 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.793 [api_impl.cc:1120] 619423 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.843 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.890 [stream.cc:2246] 619423 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.934 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.842.949 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.266 [npu_driver.cc:6125] 619423 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.318 [npu_driver.cc:6137] 619423 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=411, streamId=3, logicCq=411, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.340 [npu_driver.cc:6189] 619423 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=3, logicCq=411. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.356 [npu_driver.cc:5916] 619423 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc5c2000, addr=dfffcc5c2000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.368 [stream.cc:653] 619423 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619423,python3.9):2025-04-03-09:35:43.843.416 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.432 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.843.765 [stream.cc:291] 619423 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.425 [stream.cc:688] 619423 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=1, streamResId=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.446 [stream_state_callback_manager.cc:45] 619423 Notify: not reg stream state callback func! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.456 [api_impl.cc:1162] 619423 StreamCreate: Succ, flags=1536, stream_id=3, context=aaaaf7bf5f00 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.468 [stream.cpp:189]619423 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.483 [stream.cc:3161] 619423 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.492 [stream.cpp:195]619423 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.844.524 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaaf8c90f80. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.653 [callback.cpp:25]619423 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471981645728. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.685 [npu_driver.cc:462] 619423 EventIdAlloc: event id alloc begin, device_id=1, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.710 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.746 [npu_driver.cc:483] 619423 EventIdAlloc: Event id alloc success, device_id=1, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.755 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.760 [event.cc:347] 619423 GenEventId: Event id alloc success, device_id=1, tsId=0, eventFlag=1, event_id=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.771 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.788 [subscribe.cc:54] 619423 AssignGroupId: give groupId=320 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.801 [npu_driver.cc:3775] 619423 SqCqAllocate: deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.822 [npu_driver.cc:3785] 619423 SqCqAllocate: alloc success, deviceId=1 sqId=133, cqId=133. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.837 [subscribe.cc:131] 619423 Insert: callback subscribe: groupId=320, threadId=281471981645728, devId=1, tsId=0, streamId=2, sqId=133, cqId=133 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.851 [callback.cpp:31]619423 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471981645728. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.857 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.844.874 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaaf8b5f1c0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.901 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.924 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.957 [callback.cpp:25]619423 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471973191584. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.978 [npu_driver.cc:462] 619423 EventIdAlloc: event id alloc begin, device_id=1, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.844.987 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.844.996 [npu_driver.cc:483] 619423 EventIdAlloc: Event id alloc success, device_id=1, ts_id=0, event_id=1, eventFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.022 [event.cc:347] 619423 GenEventId: Event id alloc success, device_id=1, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.023 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.035 [subscribe.cc:54] 619423 AssignGroupId: give groupId=321 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.039 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.046 [npu_driver.cc:3775] 619423 SqCqAllocate: deviceId=1, tsId=0, groupId=321. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.845.053 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.060 [npu_driver.cc:3785] 619423 SqCqAllocate: alloc success, deviceId=1 sqId=134, cqId=134. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.845.064 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.072 [subscribe.cc:131] 619423 Insert: callback subscribe: groupId=321, threadId=281471973191584, devId=1, tsId=0, streamId=3, sqId=134, cqId=134 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.845.088 [callback.cpp:31]619423 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471973191584. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.094 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.110 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaaf8c90f80. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.133 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.218 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.595 [npu_driver.cc:2616] 619423 MemGetInfoEx: memory info type=1, type=2, free=8863793152, hug_free=56245616640 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.636 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.708 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.750 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.777 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.796 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62093M, Reserved MOC size for Other Components(HCCL/rts/etc.):3891M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58202M. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.845.816 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.845.839 [event.cpp:260]619423 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.845.858 [api_impl.cc:5713] 619423 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.846.933 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.846.951 [timeout_set_task.cc:115] 619423 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.846.964 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.846.976 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.846.993 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.004 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.017 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.037 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.076 [stars_engine.cc:622] 619423 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=409, retCode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.093 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.104 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.114 [stream.cc:2886] 619423 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.126 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.137 [stream.cc:2886] 619423 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.148 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.160 [stream.cc:2886] 619423 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.171 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.182 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.192 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.202 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.218 [stars_engine.cc:437] 619423 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.230 [stream.cc:2886] 619423 StarsWaitForTask: No.4 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.240 [event.cpp:262]619423 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.253 [event.cpp:269]619423 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.271 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.284 [timeout_set_task.cc:115] 619423 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.293 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.304 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.320 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.331 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.343 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.354 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.366 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.385 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.398 [stream.cc:2886] 619423 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.408 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.419 [stream.cc:2886] 619423 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.430 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.441 [stream.cc:2886] 619423 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.451 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.464 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.476 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.486 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.499 [stars_engine.cc:437] 619423 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.510 [stream.cc:2886] 619423 StarsWaitForTask: No.4 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.520 [event.cpp:277]619423 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.542 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.579 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.594 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 1 has been set [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.608 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.622 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.667 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.699 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619423,ffffb5717020,python3.9):2025-04-03-09:35:43.847.750 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.866 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.885 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.915 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.953 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:43.847.968 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.847.978 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.848.117 [acl.cpp:288]619423 aclInit: start to execute aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.848.162 [acl.cpp:298]619423 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.848.276 [acl.cpp:242]619423 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.857.277 [error_manager.cc:673]619423 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:43.857.937 [acl.cpp:315]619423 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:43.858.020 [prof_atls_plugin.cpp:210] (tid:619423) Module[45] register callback of ctrl handle. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.858.067 [ge_executor.cc:273]619423 Initialize:Init GeExecutor begin. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.858.340 [op_lib_register.cc:117]619423 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.428 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.468 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.502 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.541 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.572 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.602 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.640 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.668 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.697 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.723 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.750 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.778 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.804 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.832 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.859 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.888 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.916 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.945 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.858.978 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.859.012 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.058 [op_lib_register.cc:165]619423 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.096 [plugin_manager.cc:616]619423 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.113 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.144 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.166 [plugin_manager.cc:624]619423 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.181 [plugin_manager.cc:206]619423 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.811 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.859.850 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.859.865 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.859.876 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.859.917 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.278 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.327 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.341 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.352 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.386 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.423 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.437 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.447 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.460 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.473 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.821 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.852 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.867 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.860.876 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.860.910 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.249 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.281 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.295 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.308 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.343 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.385 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.399 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.411 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.423 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.436 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.773 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.803 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.815 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.861.826 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.861.857 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.194 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.224 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.237 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.248 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.281 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.312 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.324 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.342 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.355 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.367 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.699 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.729 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.742 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.862.752 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.862.784 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.119 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.148 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.161 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.172 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.204 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.235 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.247 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.259 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.269 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.281 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.627 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.657 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.670 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.863.680 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.863.712 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.049 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.078 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.092 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.103 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.133 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.179 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.194 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.205 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.216 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.228 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.239 [plugin_manager.cc:218]619423 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.264 [plugin_manager.cc:422]619423 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.275 [plugin_manager.cc:155]619423 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.295 [plugin_manager.cc:158]619423 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.306 [plugin_manager.cc:426]619423 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.637 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.668 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.680 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.864.690 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.725 [plugin_manager.cc:439]619423 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.743 [plugin_manager.cc:175]619423 ReversePathString:Enter ReversePathString schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:43.864.764 [plugin_manager.cc:187]619423 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619423,python3.9):2025-04-03-09:35:43.865.087 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:43.865.124 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:43.865.143 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619423,python3.9):2025-04-03-09:35:43.865.186 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619423,python3.9):2025-04-03-09:35:43.880.986 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.881.595 [op_tiling_registry.cc:162]619423 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619423,python3.9):2025-04-03-09:35:43.881.713 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.881.813 [op_tiling_registry.cc:162]619423 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619423,python3.9):2025-04-03-09:35:43.881.917 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.017 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.111 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.222 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.239 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.257 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.348 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.441 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.529 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.622 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619423,python3.9):2025-04-03-09:35:43.882.718 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.045 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.132 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.237 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.336 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.433 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.530 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.638 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.818 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619423,python3.9):2025-04-03-09:35:43.883.917 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.021 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.118 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.135 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.248 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.364 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.380 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.471 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.569 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.663 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.884.860 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.022 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.114 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.211 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.311 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.726 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.831 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.885.940 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.116 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.217 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.323 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.436 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.535 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.627 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.724 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.829 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.922 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619423,python3.9):2025-04-03-09:35:43.886.938 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.037 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.136 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.239 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.256 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.343 [op_tiling_registry.cc:122]619423 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.439 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.538 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.634 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.743 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619423,python3.9):2025-04-03-09:35:43.887.928 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.016 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.190 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.264 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.338 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.409 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.523 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.570 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.593 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.718 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.794 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.867 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.888.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.016 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.120 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.189 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.214 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.239 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.306 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.468 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.540 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.611 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.684 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.918 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.889.990 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.091 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.166 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.242 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.394 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.466 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.890.538 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.894.613 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619423,python3.9):2025-04-03-09:35:43.894.644 [op_tiling_registry.cc:148]619423 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619423,python3.9):2025-04-03-09:35:43.895.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.099 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.226 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.495 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.616 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.832 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.896.977 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.084 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.375 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.488 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.603 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.841 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.953 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.897.981 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.130 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.379 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.496 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.608 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.637 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.744 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.855 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.898.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.113 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.219 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.334 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.453 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.677 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.786 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.899.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.008 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.045 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.264 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.523 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.718 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.900.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.031 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.179 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.289 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.315 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.556 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.664 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.689 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.799 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.901.907 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.012 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.121 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.316 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.343 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.446 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.554 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.751 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.858 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.902.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.087 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.199 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.382 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.530 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.675 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.826 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.903.948 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.053 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.180 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.290 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.400 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.652 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.824 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.904.879 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff21345d70, registered count:1 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.047 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.070 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff2134e248, registered count:2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.194 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.440 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.550 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.674 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.905.897 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.121 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.228 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.253 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.366 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.472 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.586 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.691 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.796 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.906.914 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.018 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.126 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.268 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.317 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.339 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.492 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.604 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.712 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.818 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.907.933 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.038 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.063 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.086 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.216 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.324 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.432 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.616 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.731 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.836 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.908.946 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.063 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.172 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.283 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.311 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.417 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.522 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.552 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.823 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.909.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.036 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.150 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.256 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.378 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.487 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.591 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.784 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.910.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.911.020 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.913.599 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.913.659 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.913.684 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.913.816 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.913.933 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.043 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.149 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.254 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.360 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.475 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.579 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.699 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.726 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.843 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.869 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.914.986 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.012 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.230 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.340 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.444 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.658 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.915.925 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.916.073 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.916.202 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.916.327 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.916.439 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.917.071 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.917.847 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.918.153 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.918.582 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.162 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.283 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.395 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.539 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.655 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.919.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.922.092 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.922.903 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.923.065 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.923.228 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.924.194 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.924.348 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.924.503 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.924.952 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.065 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.175 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.281 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.390 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.502 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.615 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.925.948 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.069 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.222 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.333 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.450 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.629 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.745 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.926.973 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.122 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.424 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.561 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.842 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.927.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.260 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.420 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.530 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.643 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.755 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.780 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.890 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.928.917 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.017 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.153 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.270 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.379 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.494 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.666 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.789 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.929.980 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.122 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.471 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.647 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.828 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.930.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.058 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.168 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.276 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.306 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.598 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.931.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.932.047 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.932.155 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.932.198 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.932.320 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.932.544 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.933.794 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.936.288 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.936.506 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.936.710 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.937.020 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.937.185 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.937.339 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.153 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.318 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.744 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.938.882 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.037 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.191 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.331 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.473 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.686 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.939.992 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.022 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.048 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.074 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.130 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.154 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.190 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.211 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.232 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.254 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.277 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.298 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.322 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.350 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.373 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.394 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.416 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.437 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.459 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.479 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.505 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.571 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.594 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.616 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.637 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.685 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.705 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.747 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.789 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.811 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.831 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.851 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.874 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.896 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.919 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.963 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.940.985 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.027 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.048 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.070 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.094 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.117 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.165 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.185 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.205 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.225 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.411 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.734 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.762 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.786 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.807 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.941.971 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.082 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.196 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.308 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.423 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.533 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.697 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.816 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.942.978 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.224 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.253 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.281 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.399 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.426 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.452 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.559 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.711 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.943.886 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.031 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.149 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.470 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.666 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.785 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.899 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.927 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.944.952 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.117 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.445 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.596 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.786 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.945.949 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.071 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.230 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.397 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.577 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.946.943 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.066 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.182 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.342 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.496 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.613 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.730 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.847 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.947.973 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.948.088 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.948.542 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.948.996 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.121 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.266 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.387 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.511 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.621 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.743 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.854 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.949.972 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.096 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.209 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.371 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.606 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.798 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.950.912 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.041 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.153 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.353 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.534 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.656 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.951.890 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.015 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.261 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.387 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.546 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.952.824 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.953.083 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.953.278 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.953.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.953.637 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.953.859 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.077 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.333 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.355 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.376 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.405 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.954.471 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.955.221 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.955.642 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.956.068 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.956.280 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.956.489 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.956.919 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.957.056 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.957.229 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.960.003 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.961.847 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.961.995 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.962.115 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.962.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.962.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.962.688 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.962.876 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.216 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.248 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.273 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.320 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.510 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.537 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.963.879 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.045 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.222 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.464 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.621 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.796 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.964.967 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.149 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.315 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.477 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.651 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.842 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.870 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.965.986 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.015 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.137 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.164 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.288 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.315 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.435 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.556 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.681 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.807 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.966.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.967.036 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.967.240 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.967.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.967.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.968.227 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.968.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.968.768 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.968.957 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.149 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.256 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.286 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.392 [op_tiling_registry.cc:134]619423 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.422 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.567 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.594 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.618 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.969.644 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.970.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.194 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.349 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.503 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.657 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.695 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.971.986 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.130 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.171 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.331 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.484 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.635 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.803 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.972.955 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.973.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.973.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.973.949 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.974.140 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.975.198 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.975.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.201 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.367 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.647 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.960 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.976.990 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.112 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.475 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.977.915 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.044 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.545 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.692 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.816 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.978.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.098 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.273 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.448 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.477 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.636 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.979.823 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.980.007 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.981.564 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.987.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.987.387 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.990.225 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.990.498 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.990.796 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.991.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.991.333 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.991.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.991.698 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.991.850 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.033 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.187 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.277 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.530 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.657 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.992.808 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.275 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.584 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.616 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.729 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.993.859 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.038 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.597 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.716 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.831 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.994.859 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.995.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.995.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.019 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.299 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.333 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.460 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.571 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.597 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.622 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.768 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.996.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.046 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.286 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.477 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.505 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.529 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:43.997.764 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.520 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.607 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.630 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.698 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.773 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.793 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.000.866 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.016 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.034 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.142 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.162 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.180 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.200 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.362 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.001.382 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.002.546 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff21c98a10, registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.002.582 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff21c98c70, registered count:1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.002.996 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.003.481 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.005.165 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.005.332 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.005.415 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.005.802 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.006.273 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.317 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.437 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.844 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.037.985 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.038.598 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.038.683 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.038.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.039.009 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.039.167 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.039.400 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.039.565 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.039.967 [op_tiling_manager.cc:79]619423 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.029 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.242 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.291 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.356 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.420 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.462 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.511 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.040.558 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.454 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.491 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.511 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.528 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.548 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.884 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff544499b0, registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.909 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.926 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.941 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.041.956 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.281 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff5444cda0, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.307 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.323 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.338 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.353 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.042.653 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.042.714 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.042.813 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.592 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff544032c0, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.621 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.637 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.653 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.667 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.043.991 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff54407978, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.011 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.027 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.043 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.066 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.371 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.424 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.504 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.553 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.597 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.642 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.676 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.713 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.751 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.044.784 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.819 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.863 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.901 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.937 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.044.975 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.004 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.039 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.074 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.104 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.139 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.174 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.222 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.257 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.292 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.323 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.359 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.393 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.423 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.458 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.491 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.524 [op_tiling_manager.cc:73]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.565 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.604 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.635 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.670 [op_tiling_manager.cc:58]619423 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.045.706 [op_tiling_manager.cc:67]619423 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.734 [op_tiling_manager.cc:75]619423 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.750 [op_tiling_manager.cc:109]619423 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [186655] micro seconds. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.878 [plugin_manager.cc:639]619423 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.894 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.918 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.045.978 [host_cpu_engine.cc:254]619423 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.051.999 [host_cpu_engine.cc:273]619423 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619423,python3.9):2025-04-03-09:35:44.052.064 [host_cpu_engine.cc:254]619423 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.058.036 [host_cpu_engine.cc:267]619423 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619423,python3.9):2025-04-03-09:35:44.058.817 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619423] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.846 [folding.cc:404][InitCpuConstantFoldingNew][tid:619423]:"Init cpu constant folding begin." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.882 [folding.cc:408][InitCpuConstantFoldingNew][tid:619423]:"Number of registered ops: 234" [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.905 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Acos]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.923 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Acosh]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.935 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.948 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.961 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.971 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.983 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.058.994 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Add]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.005 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Addcdiv]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.017 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Addcmul]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.028 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[AffineGrid]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.039 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ApplyAdamV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.051 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Asin]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.060 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Asinh]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.071 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Assert]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.083 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Atan]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.094 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Atan2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.106 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Atanh]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.118 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[AvgPoolV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.128 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Betainc]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.138 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Bucketize]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.149 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CTCLossV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.160 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.172 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CacheSwapTable]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.183 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.194 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CaseCondition]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.205 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cast]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.216 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Ceil]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.232 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CheckNumerics]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.244 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ClipByValue]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.255 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ClipByValueV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.266 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Combinations]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.277 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CompareAndBitpack]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.289 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.301 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ConcatV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.311 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Conj]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.323 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ConjugateTranspose]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.334 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.345 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cosh]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[CropAndResize]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.366 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cross]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.377 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cummax]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.388 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cummin]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.399 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Cumsum]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.409 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Div]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.420 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[DynamicStitch]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.430 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EditDistance]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.442 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Elu]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.455 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EluGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.466 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.480 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.492 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.515 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.526 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.536 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.546 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.566 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.578 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.591 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.603 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.615 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.627 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.638 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingLookup]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.649 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.663 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.672 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.685 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.696 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.708 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Equal]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.717 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Expand]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.729 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ExpandDims]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.739 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Exponential]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.750 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.761 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Fill]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.772 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[FillDiagonal]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.783 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Fills]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.794 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Floor]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.805 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[FloorDiv]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.816 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[FloorMod]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.826 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GEMM]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.838 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GatherElements]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.848 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GatherV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.861 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Geometric]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.877 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GetDynamicDims]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.888 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Greater]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.899 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GreaterEqual]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.910 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GridSampler2D]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.921 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.932 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GridSampler3D]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.943 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.954 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Histogram]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.965 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.976 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Identity]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.987 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[IndexToAddr]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.059.998 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.009 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InitPartitionMap]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.020 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.031 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.045 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InvGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.057 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[InvertPermutation]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.067 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[IsClose]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.077 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[IsInf]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.087 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[IsNan]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.098 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Less]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.109 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[LinSpace]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.120 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[LinearSumAssignment]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.131 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Log]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.142 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Log1p]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.153 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.178 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.188 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.200 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MaskedFill]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.216 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MaskedScatter]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MaskedSelect]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.240 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.251 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MatMul]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.262 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.273 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MatrixDiagV3]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MinAreaPolygons]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.297 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Mul]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.307 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.318 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.329 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.340 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NLLLoss]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.351 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NLLLossGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.362 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NMSWithMask]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.373 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Neg]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NonZero]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.408 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.418 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.429 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[NotEqual]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.440 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[OnesLike]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.452 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[PSAMask]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.462 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[PSAMaskGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.474 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Pad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.485 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[PadD]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.496 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[PadV3]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.506 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[PadV3Grad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.517 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Pinverse]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Poisson]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.544 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Pow]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.556 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.567 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RNNTLoss]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.578 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RandomStandardNormal]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.589 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RandomUniform]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.601 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RealDiv]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.611 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ReduceAll]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.622 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ReduceMean]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.634 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ReduceSum]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.645 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Reshape]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.656 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Resize]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.667 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ResizeBilinear]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.677 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.688 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.699 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ResizeGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.710 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ResizeTrilinear]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.720 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ReverseSequence]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.732 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RightShift]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.743 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.754 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[RotatedNMS]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.764 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Round]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.775 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Rsqrt]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.789 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[STFT]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.801 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.812 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ScatterElements]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.823 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ScatterNdAdd]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.834 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ScatterNdMax]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.845 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ScatterNdMin]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.860 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SearchSorted]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.871 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SelectV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.884 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Sigmoid]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.895 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SigmoidGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.907 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Sign]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.919 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SilentCheck]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.931 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SilentCheckV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.941 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SilentCheckV3]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.954 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SliceWrite]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.965 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SoftmaxV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.977 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Sort]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.987 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.060.999 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SparseSegmentMean]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.010 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.021 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SparseSegmentSum]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.032 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SparseToDense]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.043 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SpatialTransformer]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.054 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SplitD]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.066 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SplitV]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.077 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Sqrt]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.088 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SqrtGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.098 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Square]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.109 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[SquaredDifference]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.120 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[StatelessBernoulli]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.131 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.142 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[StatelessRandperm]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.153 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[StridedSlice]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.164 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[StridedSliceV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.180 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Sub]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.191 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TableToResource]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.203 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Tan]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.215 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TanhGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TensorEqual]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.240 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.251 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ThreeNN]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.261 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Tile]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.272 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TileWithAxis]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.283 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TopK]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.294 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TopKPQDistance]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.305 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.317 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TopKV2D]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.328 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Trace]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.339 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TransData]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.350 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[TransDataRNN]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.361 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Tril]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.372 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Triu]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.394 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.405 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UninitPartitionMap]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UniqueConsecutive]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.428 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.439 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.450 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.460 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.471 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.482 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.493 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ViewCopy]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.509 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Voxelization]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.521 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[WarpAffine]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.532 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[ZerosLike]." [INFO] AICPU(619423,python3.9):2025-04-03-09:35:44.061.545 [folding.cc:413][InitCpuConstantFoldingNew][tid:619423]:"Register op[Zeta]." [INFO] OP(619423,python3.9):2025-04-03-09:35:44.061.562 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619423] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619423,python3.9):2025-04-03-09:35:44.061.576 [host_cpu_engine.cc:273]619423 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619423,python3.9):2025-04-03-09:35:44.061.709 [plugin_manager.cc:872]619423 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.064.938 [plugin_manager.cc:883]619423 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.004 [ops_kernel_executor_manager.cc:37]619423 Initialize:No library to load [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.019 [plugin_manager.cc:540]619423 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.029 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.057 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.076 [plugin_manager.cc:547]619423 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.090 [plugin_manager.cc:206]619423 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.828 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.065.870 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.065.885 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.065.896 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.065.939 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.288 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.320 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.333 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.344 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.379 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.416 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.429 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.441 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.453 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.484 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.497 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.507 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.518 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.854 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.885 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.898 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.066.909 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.066.944 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.295 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.327 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.340 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.352 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.387 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.421 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.435 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.446 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.457 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.485 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.496 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.507 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.517 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.850 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.880 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.893 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.067.904 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.067.947 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.298 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.335 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.349 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.361 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.395 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.429 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.442 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.455 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.466 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.495 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.506 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.517 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.530 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.865 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.896 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.910 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.068.923 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.068.966 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.306 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.338 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.352 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.364 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.397 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.431 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.445 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.457 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.469 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.499 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.510 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.521 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.535 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.870 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.901 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.916 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.069.938 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.069.973 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.312 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.346 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.361 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.372 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.404 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.436 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.449 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.462 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.473 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.501 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.512 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.523 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.535 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.547 [plugin_manager.cc:218]619423 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.577 [plugin_manager.cc:422]619423 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.591 [plugin_manager.cc:155]619423 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.611 [plugin_manager.cc:158]619423 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.623 [plugin_manager.cc:426]619423 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.070.951 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.982 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.070.996 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.071.006 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.038 [plugin_manager.cc:439]619423 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.058 [ge_executor.cc:259]619423 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.423 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.463 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.483 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.579 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.594 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.668 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.682 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.071.712 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.756 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.767 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.071.795 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.837 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.851 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.912 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.924 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.984 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.071.996 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.023 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.060 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.076 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.104 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.141 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.153 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.227 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.242 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.305 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.317 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.343 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.381 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.395 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.422 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.459 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.470 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.533 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.545 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.606 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.619 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.651 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.689 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.702 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.727 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.764 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.777 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.837 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.849 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.909 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.921 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.072.946 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.984 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.072.995 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.073.021 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.058 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.071 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.073.098 [opsproto_manager.cc:123]619423 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.139 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.201 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.213 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.223 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.233 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.073.245 [opsproto_manager.cc:147]619423 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.586 [operator_factory_impl.cc:215]619423 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.608 [operator_factory_impl.cc:269]619423 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619423,python3.9):2025-04-03-09:35:44.073.639 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.079.596 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.390 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.488 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.532 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.562 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.591 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.620 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.646 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.671 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.080.708 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.101.875 [operator_factory_impl.cc:368]619423 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619423,python3.9):2025-04-03-09:35:44.102.088 [operator_factory_impl.cc:284]619423 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.255 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.277 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.290 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.301 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.313 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619423,python3.9):2025-04-03-09:35:44.102.332 [operator_factory_impl.cc:254]619423 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619423,python3.9):2025-04-03-09:35:44.102.355 [operator_factory_impl.cc:340]619423 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619423,python3.9):2025-04-03-09:35:44.102.378 [operator_factory_impl.cc:307]619423 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.405 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.422 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.438 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.102.548 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.363 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.394 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.411 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.428 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.445 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.458 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.475 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.509 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.525 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.552 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.567 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.587 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.599 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.627 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.657 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.674 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.737 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.781 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.798 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.812 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.839 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.103.860 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.032 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.059 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.075 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.095 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.110 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.125 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.141 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.176 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.188 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.202 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.297 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.314 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.328 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.349 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.361 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.372 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.687 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.705 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.719 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.732 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.743 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.755 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.768 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.791 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.804 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.817 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.829 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.850 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.105.865 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.096 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.114 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.168 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.183 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.194 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.206 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.106.217 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.135 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.161 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.184 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.207 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.220 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.233 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.246 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.355 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.440 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.809 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.829 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.842 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.852 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.861 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.107.872 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.343 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.675 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.699 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.712 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.726 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.108.887 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.038 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.317 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.343 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.379 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.393 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.109.965 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.012 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.028 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.050 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.066 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.086 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.105 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.122 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.174 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.212 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.606 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.110.797 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.111.045 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.111.076 [operator_factory_impl.cc:222]619423 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.674 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.715 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.739 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.763 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.789 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.812 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.834 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.854 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.877 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.922 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.941 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.972 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.111.998 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.021 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.043 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.065 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.085 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.105 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.143 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.179 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.196 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.214 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.257 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.281 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.330 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.358 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.398 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.418 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.440 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.460 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.481 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.552 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.573 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.595 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.616 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.638 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.661 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.683 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.705 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.725 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.745 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.811 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.845 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.867 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.889 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.913 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.935 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.955 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.978 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.112.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.023 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.044 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.068 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.088 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.129 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.152 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.173 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.193 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.216 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.256 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.280 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.301 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.323 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.370 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.393 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.416 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.438 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.458 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.476 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.497 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.513 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.559 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.581 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.607 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.627 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.670 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.691 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.714 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.734 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.756 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.777 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.800 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.822 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.845 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.865 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.889 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.914 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.936 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.957 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.113.981 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.004 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.024 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.046 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.068 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.088 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.108 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.128 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.156 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.178 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.198 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.219 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.262 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.324 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.347 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.366 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.386 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.410 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.431 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.486 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.549 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.573 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.595 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.617 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.638 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.655 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.682 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.702 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.741 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.779 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.797 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.817 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.932 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.959 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.114.985 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.008 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.032 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.054 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.078 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.100 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.121 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.164 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.184 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.205 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.225 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.271 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.340 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.383 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.425 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.470 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.513 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.536 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.560 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.582 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.603 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.645 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.667 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.689 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.709 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.748 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.772 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.795 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.816 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.870 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.893 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.913 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.935 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.957 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.115.981 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.001 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.021 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.042 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.067 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.090 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.112 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.147 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.183 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.208 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.229 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.253 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.275 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.317 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.340 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.379 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.398 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.425 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.445 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.466 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.513 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.533 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.555 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.574 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.595 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.703 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.806 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.116.921 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.030 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.148 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.173 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.201 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.221 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.322 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.532 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.647 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.748 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.863 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.117.984 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.007 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.029 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.051 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.150 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.258 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.323 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.363 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.383 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.477 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.577 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.690 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.791 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.814 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.118.920 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.029 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.053 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.071 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.090 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.187 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.405 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.506 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.529 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.633 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.734 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.760 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.854 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.119.953 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.057 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.168 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.192 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.299 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.406 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.511 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.609 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.824 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.853 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.924 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.946 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.120.971 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.002 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.028 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.056 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.077 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.194 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.318 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.337 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.353 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.370 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.387 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.480 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.579 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.683 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.819 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.923 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.951 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.121.973 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.067 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.167 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.390 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.496 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.603 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.798 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.896 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.122.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.102 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.127 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.148 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.357 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.476 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.584 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.681 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.782 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.882 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.123.906 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.113 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.231 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.337 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.361 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.455 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.549 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.645 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.960 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.124.983 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.098 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.120 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.138 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.155 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.172 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.189 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.205 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.220 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.237 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.254 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.270 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.286 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.303 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.338 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.358 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.375 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.393 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.490 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.589 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.802 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.125.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.002 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.102 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.130 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.154 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.173 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.271 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.298 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.326 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.353 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.473 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.491 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.509 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.606 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.726 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.824 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.126.922 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.033 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.228 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.564 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.662 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.773 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.876 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.906 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.127.954 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.048 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.146 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.469 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.666 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.128.879 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.001 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.109 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.217 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.318 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.420 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.448 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.546 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.648 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.672 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.690 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.760 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.778 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.796 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.893 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.129.920 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.018 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.115 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.210 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.322 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.540 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.644 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.675 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.872 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.130.967 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.068 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.172 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.274 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.411 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.475 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.500 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.522 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.542 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.562 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.584 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.606 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.628 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.651 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.672 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.693 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.715 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.735 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.764 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.131.903 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.010 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.037 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.138 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.250 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.445 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.557 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.850 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.132.952 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.061 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.173 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.391 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.497 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.600 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.720 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.746 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.851 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.133.971 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.073 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.181 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.323 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.438 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.544 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.651 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.134.975 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.081 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.182 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.292 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.617 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.750 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.858 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.135.965 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.083 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.343 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.556 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.770 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.886 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.136.994 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.239 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.469 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.573 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.692 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.800 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.137.963 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.086 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.200 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.412 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.520 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.636 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.751 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.871 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.138.985 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.011 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.119 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.227 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.344 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.371 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.395 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.415 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.439 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.462 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.482 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.503 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.520 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.623 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.670 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.775 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.139.990 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.226 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.278 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.299 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.400 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.508 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.641 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.741 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.857 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.962 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.140.987 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.008 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.033 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.136 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.165 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.190 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.287 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.313 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.335 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.439 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.464 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.486 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.509 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.620 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.729 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.835 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.883 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.141.993 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.019 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.119 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.226 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.336 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.484 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.508 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.636 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.740 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.843 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.142.949 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.064 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.167 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.270 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.387 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.503 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.609 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.727 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.143.935 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.044 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.150 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.280 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.331 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.466 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.488 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.591 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.807 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.144.918 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.039 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.148 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.259 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.365 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.474 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.589 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.626 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.670 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.145.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.029 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.143 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.259 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.366 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.477 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.583 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.687 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.802 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.146.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.016 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.122 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.232 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.456 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.564 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.666 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.147.878 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.343 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.546 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.664 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.689 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.794 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.148.901 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.011 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.248 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.369 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.602 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.824 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.149.932 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.042 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.150 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.259 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.368 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.481 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.597 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.706 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.819 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.150.926 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.048 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.156 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.262 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.369 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.594 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.151.802 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.011 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.418 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.483 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.512 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.542 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.572 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.607 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.627 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.664 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.695 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.716 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.821 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.152.931 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.044 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.249 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.360 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.461 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.552 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.587 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.701 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.805 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.153.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.038 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.248 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.278 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.313 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.333 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.353 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.528 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.559 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.595 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.627 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.692 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.756 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.782 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.838 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.862 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.900 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.934 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.962 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.154.990 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.027 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.107 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.213 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.465 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.575 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.684 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.790 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.893 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.155.920 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.031 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.253 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.363 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.388 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.592 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.695 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.809 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.886 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.156.975 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.096 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.203 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.310 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.428 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.541 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.623 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.724 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.807 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.157.914 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.019 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.117 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.219 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.327 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.469 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.573 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.679 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.787 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.158.895 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.002 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.251 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.356 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.479 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.585 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.721 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.754 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.784 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.812 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.840 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.159.906 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.009 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.113 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.211 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.244 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.269 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.375 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.478 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.584 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.688 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.789 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.902 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.160.954 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.100 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.128 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.221 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.327 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.429 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.539 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.652 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.756 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.863 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.161.970 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.074 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.152 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.183 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.289 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.397 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.516 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.617 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.725 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.833 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.162.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.022 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.050 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.269 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.596 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.858 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.885 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.918 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.944 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.970 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.163.994 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.019 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.042 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.061 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.083 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.106 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.172 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.297 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.500 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.621 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.678 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.781 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.892 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.164.997 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.220 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.244 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.267 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.290 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.312 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.415 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.634 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.739 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.768 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.795 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.165.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.041 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.150 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.264 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.369 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.471 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.499 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.605 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.777 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.805 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.833 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.864 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.892 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.918 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.947 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.972 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.166.998 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.024 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.052 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.079 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.103 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.126 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.259 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.396 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.424 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.453 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.482 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.508 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.532 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.567 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.593 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.620 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.167.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.168.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.168.865 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.168.954 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.033 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.120 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.195 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.310 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.420 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.528 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.610 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.684 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.770 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.796 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.169.927 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.005 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.139 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.218 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.381 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.715 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.741 [opsproto_manager.cc:162]619423 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.803 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.170.857 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.486 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.541 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.564 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.646 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.664 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.727 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.744 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.758 [opp_so_manager.cc:56]619423 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.773 [plugin_manager.cc:540]619423 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.787 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.816 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.836 [plugin_manager.cc:547]619423 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.171.856 [plugin_manager.cc:206]619423 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.172.241 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.293 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.312 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.328 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.172.373 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.172.731 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.765 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.782 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.796 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.172.832 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.874 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.893 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.909 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.172.924 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.956 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.974 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.172.990 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.173.006 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.173.355 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.404 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.423 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.439 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.173.476 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.173.824 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.861 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.878 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.894 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.173.935 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.981 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.173.998 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.013 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.174.027 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.063 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.079 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.091 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.174.107 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.174.455 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.504 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.523 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.537 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.174.577 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.174.938 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.974 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.174.992 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.005 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.175.040 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.084 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.101 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.114 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.175.128 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.161 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.177 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.191 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.175.208 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.175.574 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.610 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.626 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.175.641 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.175.675 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.031 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.064 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.081 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.096 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.130 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.184 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.207 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.226 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.240 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.273 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.289 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.305 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.320 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.690 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.725 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.743 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.176.757 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.176.792 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.156 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.192 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.209 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.224 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.258 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.301 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.318 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.333 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.350 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.388 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.405 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.420 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.435 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.464 [plugin_manager.cc:218]619423 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.487 [plugin_manager.cc:422]619423 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.502 [plugin_manager.cc:155]619423 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.525 [plugin_manager.cc:158]619423 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.540 [plugin_manager.cc:426]619423 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.882 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.917 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.936 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.177.949 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.177.985 [plugin_manager.cc:439]619423 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.311 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.341 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.364 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.488 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.507 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.178.523 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.180.892 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.180.930 [op_impl_registry_holder_manager.cc:216]619423 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.180.951 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.181.005 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.181.547 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.181.664 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.181.687 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.181.703 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.183.880 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.183.914 [op_impl_registry_holder_manager.cc:216]619423 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.183.933 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.183.971 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.387 [op_impl_space_registry.cc:80]619423 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.413 [op_impl_space_registry.cc:83]619423 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.437 [op_impl_space_registry.cc:80]619423 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.464 [op_impl_space_registry.cc:83]619423 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619423,python3.9):2025-04-03-09:35:44.184.482 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.534 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.184.570 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.184.661 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.184.679 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.184.694 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.185.430 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.185.456 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.185.556 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.185.576 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.185.590 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.186.217 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.186.240 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.186.283 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.186.319 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.186.418 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.186.437 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.186.452 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.187.051 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.187.075 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.187.170 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.187.189 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.187.202 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.187.801 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.187.824 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.187.870 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.187.905 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.003 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.021 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.034 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.646 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.188.672 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.796 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.816 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.188.831 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.189.420 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.189.443 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.189.490 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.189.531 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.189.628 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.189.647 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.189.661 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.190.238 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.190.262 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619423,python3.9):2025-04-03-09:35:44.190.358 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.190.379 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.190.393 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.190.972 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.190.995 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.191.058 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.191.096 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.191.138 [opp_so_manager.cc:97]619423 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.191.217 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.191.267 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.191.284 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.191.298 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.507 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.529 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.570 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.612 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.634 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.656 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.677 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.698 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.719 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.738 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.775 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.806 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.826 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.845 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.862 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.881 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.900 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.919 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.937 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.957 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.978 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.217.998 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.017 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.036 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.055 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.076 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.095 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.114 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.133 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.152 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.170 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.187 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.206 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.226 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.244 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.265 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.330 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.368 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.388 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.409 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.430 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.470 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.486 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.506 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.547 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.568 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.588 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.607 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.626 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.645 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.668 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.686 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.705 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.726 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.746 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.765 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.784 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.800 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.818 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.835 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.861 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.881 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.901 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.921 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.939 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.218.995 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.014 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.032 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.051 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.071 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.090 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.108 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.128 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.146 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.166 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.186 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.206 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.227 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.266 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.286 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.306 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.327 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.347 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.366 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.390 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.409 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.430 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.487 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.506 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.568 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.586 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.605 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.667 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.687 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.730 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.752 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.773 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.798 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.821 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.846 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.867 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.887 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.936 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.219.980 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.000 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.023 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.045 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.065 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.086 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.109 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.132 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.180 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.204 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.228 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.250 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.271 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.292 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.313 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.339 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.383 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.408 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.431 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.455 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.563 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.591 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.613 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.643 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.683 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.704 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.726 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.747 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.770 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.791 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.835 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.877 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.919 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.961 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.220.982 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.003 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.024 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.050 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.075 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.118 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.142 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.163 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.184 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.205 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.235 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.256 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.278 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.299 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.320 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.342 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.363 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.406 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.428 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.448 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.490 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.510 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.552 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.579 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.601 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.619 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.643 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.665 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.685 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.711 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.779 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.802 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.848 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.869 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.890 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.959 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.221.979 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.001 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.023 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.041 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.062 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.083 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.104 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.164 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.186 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.310 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.412 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.629 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.745 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.797 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.222.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.044 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.251 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.358 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.468 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.571 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.597 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.620 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.643 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.741 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.849 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.875 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.953 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.975 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.223.997 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.103 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.214 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.325 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.452 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.555 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.676 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.723 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.747 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.840 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.224.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.045 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.169 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.277 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.373 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.398 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.604 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.806 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.831 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.225.931 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.031 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.130 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.234 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.336 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.491 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.513 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.538 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.575 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.597 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.621 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.642 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.686 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.704 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.806 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.910 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.937 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.960 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.226.982 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.005 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.028 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.221 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.322 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.347 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.554 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.581 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.603 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.705 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.803 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.227.910 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.025 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.248 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.552 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.661 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.762 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.790 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.228.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.015 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.040 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.138 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.347 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.445 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.575 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.669 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.775 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.878 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.229.977 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.003 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.101 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.200 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.311 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.415 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.517 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.620 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.673 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.795 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.821 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.846 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.867 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.891 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.959 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.230.982 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.005 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.027 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.045 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.068 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.091 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.113 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.342 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.561 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.866 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.891 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.231.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.034 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.060 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.086 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.108 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.223 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.249 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.272 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.399 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.498 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.593 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.694 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.797 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.892 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.232.994 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.108 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.311 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.673 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.696 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.794 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.897 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.233.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.101 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.199 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.301 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.401 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.498 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.606 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.709 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.810 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.234.914 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.009 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.034 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.139 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.261 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.283 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.305 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.329 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.408 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.508 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.537 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.827 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.235.932 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.032 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.280 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.376 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.581 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.682 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.782 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.236.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.003 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.025 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.047 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.069 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.091 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.122 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.169 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.192 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.213 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.239 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.261 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.303 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.324 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.367 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.394 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.498 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.594 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.619 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.711 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.814 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.909 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.237.987 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.090 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.186 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.284 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.356 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.463 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.679 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.783 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.238.982 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.082 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.184 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.210 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.305 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.406 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.511 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.610 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.733 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.841 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.239.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.036 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.133 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.253 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.364 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.567 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.668 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.865 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.240.970 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.072 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.112 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.206 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.304 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.512 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.622 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.722 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.821 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.241.920 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.018 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.117 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.220 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.318 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.343 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.439 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.538 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.635 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.742 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.846 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.242.947 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.046 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.256 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.352 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.454 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.561 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.661 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.757 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.864 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.968 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.243.994 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.089 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.308 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.335 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.360 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.384 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.407 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.428 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.448 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.470 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.591 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.615 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.639 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.736 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.837 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.244.936 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.039 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.139 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.177 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.198 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.306 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.731 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.835 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.882 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.902 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.245.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.024 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.047 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.149 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.176 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.199 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.295 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.318 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.362 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.452 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.553 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.661 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.685 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.720 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.812 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.246.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.034 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.137 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.163 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.187 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.721 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.823 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.247.929 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.026 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.123 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.235 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.350 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.547 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.745 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.854 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.248.983 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.007 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.106 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.154 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.247 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.343 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.548 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.659 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.854 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.249.951 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.053 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.151 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.181 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.204 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.229 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.326 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.422 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.621 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.818 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.250.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.028 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.129 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.226 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.325 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.430 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.530 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.637 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.836 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.251.930 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.038 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.136 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.348 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.551 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.651 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.850 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.874 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.252.967 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.065 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.173 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.275 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.386 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.484 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.578 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.682 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.783 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.885 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.253.986 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.081 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.181 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.283 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.386 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.484 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.579 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.679 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.780 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.254.981 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.077 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.175 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.270 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.370 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.471 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.569 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.682 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.754 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.783 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.810 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.835 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.884 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.908 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.938 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.961 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.255.984 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.029 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.057 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.174 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.382 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.484 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.582 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.683 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.782 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.867 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.895 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.256.993 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.092 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.192 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.301 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.509 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.536 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.558 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.580 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.601 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.723 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.780 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.808 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.832 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.858 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.883 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.908 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.933 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.957 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.257.982 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.009 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.032 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.059 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.085 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.116 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.140 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.164 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.251 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.355 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.473 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.571 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.681 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.785 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.883 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.258.979 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.101 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.214 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.313 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.412 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.438 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.625 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.720 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.823 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.897 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.259.978 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.074 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.182 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.291 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.402 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.508 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.585 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.671 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.749 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.845 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.260.949 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.057 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.136 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.240 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.266 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.459 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.561 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.662 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.761 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.261.954 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.054 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.157 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.257 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.356 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.468 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.516 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.537 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.556 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.617 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.816 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.897 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.922 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.262.946 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.039 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.240 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.336 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.437 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.538 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.563 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.586 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.743 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.263.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.014 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.119 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.227 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.331 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.429 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.618 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.733 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.827 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.264.928 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.025 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.219 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.324 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.423 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.501 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.527 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.624 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.720 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.813 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.265.915 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.013 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.039 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.061 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.082 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.103 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.123 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.144 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.165 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.189 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.211 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.231 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.268 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.290 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.312 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.417 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.510 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.604 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.629 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.654 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.856 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.266.955 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.051 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.147 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.174 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.196 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.223 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.543 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.641 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.666 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.689 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.812 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.267.917 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.025 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.124 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.233 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.336 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.363 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.455 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.483 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.581 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.609 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.633 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.658 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.677 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.700 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.721 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.742 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.764 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.784 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.807 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.828 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.850 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.870 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.892 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.268.991 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.016 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.035 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.169 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.192 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.214 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.257 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.279 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.300 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.322 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.344 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.462 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.533 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.657 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.808 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.269.881 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.179 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.251 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.406 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.482 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.550 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.618 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.730 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.834 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.270.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.271.016 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.271.091 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.271.158 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.271.180 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.271.431 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.280.373 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.284.968 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.307.087 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.307.443 [plugin_manager.cc:554]619423 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619423,python3.9):2025-04-03-09:35:44.307.468 [plugin_manager.cc:134]619423 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.307.519 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.307.538 [plugin_manager.cc:145]619423 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.307.551 [plugin_manager.cc:557]619423 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.307.567 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.307.598 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.411 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.485 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.512 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.596 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.614 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.668 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.686 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.699 [opp_so_manager.cc:66]619423 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.717 [plugin_manager.cc:616]619423 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.748 [plugin_manager.cc:108]619423 GetOppPath:Enter get opp path schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.776 [plugin_manager.cc:117]619423 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.796 [plugin_manager.cc:624]619423 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.309.813 [plugin_manager.cc:206]619423 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.251 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.295 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.313 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.327 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.375 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.728 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.765 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.782 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.795 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.834 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.876 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.893 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.310.923 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.940 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.310.961 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.303 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.339 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.356 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.371 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.406 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.749 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.785 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.803 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.818 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.857 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.894 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.911 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.311.925 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.940 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.311.958 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.326 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.367 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.386 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.403 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.437 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.784 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.820 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.837 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.850 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.886 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.925 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.942 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.312.955 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.973 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.312.987 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.313.333 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.370 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.388 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.428 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.313.466 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.313.808 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.842 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.858 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.872 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.313.908 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.946 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.963 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.313.977 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.313.993 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.314.008 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.314.349 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.383 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.401 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.416 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.314.455 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.314.797 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.847 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.866 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.882 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.314.919 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.957 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.975 [plugin_manager.cc:1026]619423 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.314.989 [plugin_manager.cc:322]619423 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.004 [plugin_manager.cc:212]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.021 [plugin_manager.cc:215]619423 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.036 [plugin_manager.cc:218]619423 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.056 [plugin_manager.cc:422]619423 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.073 [plugin_manager.cc:155]619423 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.098 [plugin_manager.cc:158]619423 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.115 [plugin_manager.cc:426]619423 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.448 [plugin_manager.cc:250]619423 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.315.498 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.315.517 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.315.532 [plugin_manager.cc:1063]619423 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.568 [plugin_manager.cc:439]619423 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.893 [plugin_manager.cc:996]619423 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.924 [plugin_manager.cc:1011]619423 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619423,python3.9):2025-04-03-09:35:44.315.948 [plugin_manager.cc:1015]619423 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.316.008 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.316.121 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.316.140 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.316.172 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.113 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff541a02c0, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.160 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.188 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.209 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.232 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.536 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff541a4978, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.577 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.595 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.615 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.632 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.914 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.318.947 [op_impl_registry_holder_manager.cc:216]619423 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.318.966 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.319.014 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.319.379 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.319.444 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.319.567 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.319.586 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.319.603 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.329 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.376 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.399 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.420 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.453 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.753 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff541519b0, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.779 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.800 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.817 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.321.832 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.122 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff54154da0, registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.146 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.166 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.185 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.205 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.477 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.322.509 [op_impl_registry_holder_manager.cc:216]619423 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.527 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.322.573 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.047 [op_impl_space_registry.cc:84]619423 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.071 [op_impl_space_registry.cc:85]619423 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.086 [op_impl_space_registry.cc:86]619423 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.100 [op_impl_space_registry.cc:87]619423 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.134 [op_impl_space_registry.cc:93]619423 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.156 [op_impl_space_registry.cc:84]619423 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.171 [op_impl_space_registry.cc:85]619423 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.185 [op_impl_space_registry.cc:86]619423 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.200 [op_impl_space_registry.cc:87]619423 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.211 [op_impl_space_registry.cc:93]619423 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619423,python3.9):2025-04-03-09:35:44.323.226 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.292 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.333 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.323.374 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.323.487 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.323.507 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.323.522 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.324.255 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.324.289 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.324.346 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.324.449 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.324.485 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.324.501 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.325.317 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.325.347 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.325.396 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.325.434 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.325.473 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.325.572 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.325.591 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.325.603 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.326.174 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.326.200 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.326.250 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.326.360 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.326.379 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.326.414 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.327.192 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.327.220 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.327.271 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.327.318 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.327.356 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.327.462 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.327.481 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.327.496 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.328.061 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.328.087 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.328.131 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.328.239 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.328.262 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.328.276 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.329.027 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.329.072 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.329.124 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.329.165 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.329.213 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.329.326 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.329.345 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.329.359 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.329.928 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.329.955 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.330.007 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.330.112 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.330.131 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.330.145 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.330.898 [op_impl_space_registry.cc:240]619423 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.330.927 [opp_so_manager.cc:48]619423 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.331.009 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.331.054 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.331.098 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.331.125 [opp_so_manager.cc:126]619423 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.331.161 [opp_so_manager.cc:123]619423 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.331.254 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.331.274 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.331.322 [plugin_manager.cc:1126]619423 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.331.339 [plugin_manager.cc:1130]619423 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.331.351 [opp_so_manager.cc:45]619423 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.345.196 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.345.971 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.346.038 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.346.068 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.348.766 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.356.250 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.356.257 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.356.295 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.356.304 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.343 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.345 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.369 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.389 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.356.407 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.356.408 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.356.435 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.356.435 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.459 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.356.459 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.192 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.629 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.708 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.774 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.974 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.453.997 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.067 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.090 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.112 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.132 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.243 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.307 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.388 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.453 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.522 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.588 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.611 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.674 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.695 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.716 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.777 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.847 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.454.981 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.050 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.114 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.183 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.250 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.315 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.380 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.445 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.511 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.581 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.711 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.773 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.455.836 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.106 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.241 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.637 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.747 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.862 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.461.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.131 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.381 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.645 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.869 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.462.983 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.102 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.127 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.282 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.391 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.763 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.463.902 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.010 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.155 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.293 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.416 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.650 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.869 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.464.986 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.568 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.714 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.465.996 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xffff0dbc4360, registered count:5 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.025 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.047 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.067 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.090 [op_check.cc:71]619423 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.111 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.347 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.550 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.698 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.805 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.845 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.951 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.466.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.092 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.206 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.231 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.335 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.442 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.553 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.886 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.467.988 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.093 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.314 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.424 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.636 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.754 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.468.951 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.096 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.238 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.390 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.515 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.624 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.745 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.851 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.469.961 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.220 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.381 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.407 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff0dc4ee08, registered count:7 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.578 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.600 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff0dc57298, registered count:7 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.721 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.831 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.470.980 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.092 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.203 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.326 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.433 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.544 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.651 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.784 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.471.899 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.006 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.119 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.250 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.364 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.474 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.598 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.705 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.851 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.878 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.905 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.472.928 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.069 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.183 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.407 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.515 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.623 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.672 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.786 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.473.891 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.000 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.201 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.315 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.537 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.652 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.768 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.880 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.474.907 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.021 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.171 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.442 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.554 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.782 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.890 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.475.996 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.476.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.476.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.476.430 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.476.538 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.476.662 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.224 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.271 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.296 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.419 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.535 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.642 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.763 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.875 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.479.988 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.100 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.220 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.338 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.371 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.488 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.513 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.624 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.767 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.877 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.480.984 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.087 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.297 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.579 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.719 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.481.949 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.482.063 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.482.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.483.504 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.483.809 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.484.279 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.484.868 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.484.989 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.485.102 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.485.248 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.485.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.485.634 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.487.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.488.600 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.488.759 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.488.917 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.489.871 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.022 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.174 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.604 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.730 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.838 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.490.943 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.049 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.154 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.267 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.381 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.488 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.593 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.698 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.857 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.491.962 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.071 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.267 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.375 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.485 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.592 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.734 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.492.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.031 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.160 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.328 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.437 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.549 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.837 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.493.984 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.095 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.346 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.478 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.697 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.833 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.494.940 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.047 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.216 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.334 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.523 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.665 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.495.842 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.007 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.428 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.605 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.716 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.823 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.496.932 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.039 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.064 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.236 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.451 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.673 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.788 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.895 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.497.923 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.498.024 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.498.234 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.499.215 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.501.604 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.501.809 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.502.010 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.502.321 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.502.476 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.502.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.503.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.503.596 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.503.716 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.503.860 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.015 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.142 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.302 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.458 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.595 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.714 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.820 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.504.914 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.218 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.249 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.274 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.297 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.319 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.338 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.359 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.382 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.404 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.449 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.472 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.493 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.525 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.549 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.571 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.593 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.614 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.636 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.657 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.679 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.697 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.715 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.737 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.758 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.781 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.805 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.829 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.851 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.874 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.895 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.916 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.936 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.958 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.505.999 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.020 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.046 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.075 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.097 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.120 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.141 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.163 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.185 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.207 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.229 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.247 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.267 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.289 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.310 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.331 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.355 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.374 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.395 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.419 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.438 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.587 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.738 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.904 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.932 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.955 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.506.978 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.000 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.154 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.259 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.365 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.469 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.578 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.688 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.842 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.507.955 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.099 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.228 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.348 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.375 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.399 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.505 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.532 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.559 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.814 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.508.960 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.221 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.398 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.512 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.697 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.806 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.911 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.950 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.509.973 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.266 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.420 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.563 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.739 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.881 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.510.987 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.145 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.294 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.467 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.587 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.699 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.810 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.511.918 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.027 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.196 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.346 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.458 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.565 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.668 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.781 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.512.888 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.513.306 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.513.741 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.513.862 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.513.974 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.001 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.110 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.219 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.329 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.439 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.546 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.649 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.761 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.514.873 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.022 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.135 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.427 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.534 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.646 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.761 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.515.943 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.109 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.237 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.351 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.455 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.564 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.684 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.797 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.516.906 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.051 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.161 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.312 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.550 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.517.897 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.067 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.263 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.465 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.676 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.703 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.727 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.746 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.771 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.793 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.815 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.518.839 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.519.549 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.519.936 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.520.348 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.520.540 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.520.739 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.521.140 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.521.258 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.521.422 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.524.111 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.525.912 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.043 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.160 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.265 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.532 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.681 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.526.863 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.187 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.217 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.246 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.269 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.293 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.461 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.487 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.641 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.805 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.527.964 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.116 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.310 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.340 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.492 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.660 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.817 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.528.979 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.143 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.286 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.317 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.458 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.639 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.667 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.769 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.891 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.529.917 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.030 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.057 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.172 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.285 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.398 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.512 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.613 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.715 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.530.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.531.086 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.531.440 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.531.801 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.008 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.345 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.531 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.717 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.826 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.532.944 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.533.095 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.533.122 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.533.147 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.533.175 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.534.205 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.534.630 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.534.780 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.534.921 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.072 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.101 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.241 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.383 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.537 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.565 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.707 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.535.859 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.536.004 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.536.167 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.536.313 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.536.713 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.537.063 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.537.291 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.537.474 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.538.529 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.538.934 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.539.479 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.539.638 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.539.787 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.539.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.050 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.210 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.240 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.350 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.376 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.688 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.798 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.540.942 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.091 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.204 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.519 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.690 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.847 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.541.952 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.059 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.206 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.395 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.565 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.594 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.736 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.764 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.542.911 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.543.084 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.544.583 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.550.292 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.550.341 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.553.176 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.553.441 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.553.721 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.040 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.244 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.447 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.601 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.756 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.554.948 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.555.088 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.555.171 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.555.403 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.555.520 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.555.677 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.134 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.312 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.470 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.500 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.611 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.720 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.556.903 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.080 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.263 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.410 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.526 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.638 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.663 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.557.853 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.033 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.183 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.295 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.448 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.476 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.590 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.702 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.728 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.753 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.558.900 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.082 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.169 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.400 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.600 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.627 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.655 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.830 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.559.861 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.450 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.550 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.577 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.652 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.725 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.747 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.813 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.974 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.562.995 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.101 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.125 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.147 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.168 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.323 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.563.344 [tuning_tiling_registry.cc:30]619423 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.564.458 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff0e5930b0, registered count:7 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.564.497 [op_check.cc:24]619423 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff0e593310, registered count:5 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.564.888 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.565.389 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.566.984 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.567.163 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.567.245 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.567.601 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.568.043 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.598.566 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.598.710 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.598.895 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.598.976 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.599.119 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.599.262 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.599.898 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.599.987 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.600.258 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.600.346 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.600.505 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.600.749 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.600.917 [op_impl_registry.cc:122]619423 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619423,python3.9):2025-04-03-09:35:44.601.557 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.607.513 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619423,python3.9):2025-04-03-09:35:44.609.662 [op_impl_registry_holder_manager.cc:208]619423 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.690.295 [op_impl_space_registry.cc:109]619423 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.691.617 [op_impl_space_registry.cc:84]619423 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.691.645 [op_impl_space_registry.cc:85]619423 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.691.658 [op_impl_space_registry.cc:86]619423 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.691.671 [op_impl_space_registry.cc:87]619423 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.691.703 [op_impl_space_registry.cc:93]619423 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619423,python3.9):2025-04-03-09:35:44.692.774 [op_impl_space_registry.cc:273]619423 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.250 [plugin_manager.cc:564]619423 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.275 [plugin_manager.cc:134]619423 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.693.327 [file_utils.cc:47]619423 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.693.345 [plugin_manager.cc:145]619423 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619423,python3.9):2025-04-03-09:35:44.693.357 [plugin_manager.cc:567]619423 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.419 [mem_manager.cc:31]619423 Initialize:MemManager Initialize. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.458 [rts_caching_mem_allocator.cc:131]619423 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.477 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.494 [graph_mem_allocator.cc:17]619423 Initialize:MemoryAllocator::Initialize. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.507 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.522 [graph_mem_allocator.cc:17]619423 Initialize:MemoryAllocator::Initialize. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.534 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.549 [caching_allocator.cc:391]619423 FreeBlocks:Free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.562 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.580 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.595 [caching_allocator.cc:391]619423 FreeBlocks:Free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.606 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.621 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.637 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.650 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.665 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.679 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.695 [session_scope_mem_allocator.cc:73]619423 FreeAllMemory:Free all memory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.709 [mem_manager.cc:102]619423 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.724 [session_scope_mem_allocator.cc:73]619423 FreeAllMemory:Free all memory [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.775 [dump_manager.cc:152]619423 Init:exception dump is not enabled [INFO] GE(619423,python3.9):2025-04-03-09:35:44.693.834 [ge_executor.cc:315]619423 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.693.856 [acl.cpp:321]619423 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.693.872 [dump.cpp:841]619423 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.011 [dump.cpp:759]619423 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.029 [dump.cpp:594]619423 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.040 [dump.cpp:224]619423 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.083 [dump.cpp:703]619423 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.105 [dump.cpp:771]619423 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.694.121 [dump.cpp:703]619423 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.199 [adx_datadump_server.cpp:77][tid:619423] start to do dump init [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.220 [adx_dump_record.cpp:210][tid:619423] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.235 [adx_dump_record.cpp:214][tid:619423] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.440 [adx_server_manager.cpp:149][tid:619652] server manager add component (10) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.460 [adx_dump_record.cpp:627][tid:619653] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.472 [adx_server_manager.cpp:165][tid:619652] server manager components init successfully [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.627 [adx_server_manager.cpp:203][tid:619654] Run Server(0) Process [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.686 [adx_server_manager.cpp:369][tid:619654] device up 0 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.707 [hdc_comm_opt.cpp:59][tid:619654] open device[0] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.694.815 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.694.835 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.851 [hdc_api.cpp:119][tid:619654] logDevId 0 create HDC server successfully [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.887 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.904 [adx_server_manager.cpp:369][tid:619654] device up 1 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.694.918 [hdc_comm_opt.cpp:59][tid:619654] open device[1] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.041 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.059 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.072 [hdc_api.cpp:119][tid:619654] logDevId 1 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.100 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.118 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.132 [adx_server_manager.cpp:369][tid:619654] device up 2 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.144 [hdc_comm_opt.cpp:59][tid:619654] open device[2] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.209 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.228 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.242 [hdc_api.cpp:119][tid:619654] logDevId 2 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.256 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.270 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.284 [adx_server_manager.cpp:369][tid:619654] device up 3 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.297 [hdc_comm_opt.cpp:59][tid:619654] open device[3] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.416 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.434 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.446 [hdc_api.cpp:119][tid:619654] logDevId 3 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.460 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.476 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.489 [adx_server_manager.cpp:369][tid:619654] device up 4 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.506 [hdc_comm_opt.cpp:59][tid:619654] open device[4] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.627 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.645 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.657 [hdc_api.cpp:119][tid:619654] logDevId 4 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.674 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.689 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.703 [adx_server_manager.cpp:369][tid:619654] device up 5 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.713 [hdc_comm_opt.cpp:59][tid:619654] open device[5] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.832 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.695.849 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.861 [hdc_api.cpp:119][tid:619654] logDevId 5 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.873 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.899 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.914 [adx_server_manager.cpp:369][tid:619654] device up 6 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.695.929 [hdc_comm_opt.cpp:59][tid:619654] open device[6] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.696.053 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.696.072 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.085 [hdc_api.cpp:119][tid:619654] logDevId 6 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.097 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.113 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.124 [adx_server_manager.cpp:369][tid:619654] device up 7 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.140 [hdc_comm_opt.cpp:59][tid:619654] open device[7] server[13] [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.696.279 [drv_log_user.c:623][ascend][curpid:619423,619654][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.696.300 [hdc_server.c:363][ascend][curpid:619423,619654][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.315 [hdc_api.cpp:119][tid:619654] logDevId 7 create HDC server successfully [WARNING] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.327 [adx_hdc_epoll.cpp:34][tid:619654] hdc epoll has been created. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.696.342 [adx_server_manager.cpp:111][tid:619654] create server info [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.697.561 [adx_server_manager.cpp:457][tid:619423] The server is initialized after waiting 3 times. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.697.583 [ge_executor.cc:1349]619423 SetDump:Start to set dump config [INFO] GE(619423,python3.9):2025-04-03-09:35:44.697.598 [dump_manager.cc:201]619423 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.697.637 [dump_manager.cc:261]619423 SetExceptionDump:Enable brief exception dump. [INFO] GE(619423,python3.9):2025-04-03-09:35:44.697.675 [ge_executor.cc:1355]619423 SetDump:Set dump config successfully [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.697.712 [dump_manager.cpp:146][tid:619423] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:44.699.228 [dump_manager.cpp:301][tid:619423] Register exception callback, registered: 1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.253 [dump.cpp:825]619423 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.277 [acl.cpp:327]619423 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.292 [acl.cpp:329]619423 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.307 [acl_resource_manager.cpp:248]619423 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.402 [acl_resource_manager.cpp:275]619423 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.421 [acl_resource_manager.cpp:278]619423 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.451 [acl.cpp:335]619423 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.464 [acl.cpp:336]619423 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.532 [json_parser.cpp:260]619423 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.551 [acl.cpp:343]619423 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:44.699.574 [prof_atls_plugin.cpp:210] (tid:619423) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.590 [acl.cpp:351]619423 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.602 [profiling.cpp:141]619423 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.619 [profiling.cpp:135]619423 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.683 [profiling.cpp:167]619423 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619423.json], errorCode = 0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.701 [profiling.cpp:172]619423 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619423.json] is empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.712 [profiling.cpp:176]619423 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.726 [profiling.cpp:179]619423 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.699.741 [profiling.cpp:92]619423 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:44.700.133 [ai_drv_dev_api.cpp:316] >>> (tid:619423) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:44.700.275 [msprof_callback_impl.cpp:132] >>> (tid:619423) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.296 [profiling.cpp:122]619423 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.309 [profiling.cpp:186]619423 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.322 [hash_utils.cpp:108]619423 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619423.json [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.363 [hash_utils.cpp:128]619423 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619423.json, hash: 3917491750093904922 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.381 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.420 [acl.cpp:280]619423 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.435 [acl.cpp:385]619423 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.700.454 [runtime.cc:5394] 619423 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.700.472 [api_impl.cc:7407] 619423 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.700.495 [api_impl.cc:4211] 619423 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.700.524 [acl.cpp:420]619423 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.700.700 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.700.832 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.700.901 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.700.926 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.706148] [info] [619423] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist mki_log delete old file:/home/jenkins/atb/log/atb_1952491_20250328090703.log [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:44.706.827 [prof_atls_plugin.cpp:210] (tid:619423) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.707.641 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.707.671 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.707.748 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 1, rank size = 8. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.707.769 [device.cpp:143]619423 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.796 [api_impl.cc:2999] 619423 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.814 [device.cc:306] 619423 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.833 [runtime.cc:3393] 619423 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.849 [runtime.cc:3560] 619423 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.865 [api_impl.cc:2987] 619423 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.882 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.898 [api_impl.cc:2992] 619423 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.707.920 [api_impl.cc:3025] 619423 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.707.936 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.707.948 [acl.cpp:121]619423 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.707.970 [device.cpp:154]619423 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.708.026 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.708.057 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is _npu_log, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.708.081 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.708.098 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.708.115 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.708.146 [queue_client.c:151][ascend][curpid:619423,619423][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.708.316 [queue_client.c:305][ascend][curpid:619423,619423][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=1; vfid=0; hostpid=619423; master_pid=619423; proc_type=0; devpid=31531). [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.712.380 [queue_client.c:362][ascend][curpid:619423,619423][drv][queuemng][QueueInit]QueueInit device. (devid=1; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.712.581 [npu_driver.cc:4114] 619423 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.712.606 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.712.624 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=_npu_log. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.712.914 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.712.935 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.081 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.110 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.131 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.148 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.713.160 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.183 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.206 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.220 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.236 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.277 [npu_driver.cc:4087] 619423 MemQueueInit: repeated init [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.295 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.312 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=ms_tensor_dump. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.713.713 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.749 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.938 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.713.968 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.713.992 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.713.992 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.008 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.019 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.039 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.055 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.067 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.107 [npu_driver.cc:4087] 619423 MemQueueInit: repeated init [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.125 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.138 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=ms_tensor_summary. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.714.295 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.315 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.425 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.454 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.714.466 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.478 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.485 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.493 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.504 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.518 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.545 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.581 [npu_driver.cc:4087] 619423 MemQueueInit: repeated init [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.597 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.614 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=ms_image_summary. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.714.754 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.774 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.880 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.908 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.714.922 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.933 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.714.940 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.948 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.959 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.972 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.714.985 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.025 [npu_driver.cc:4087] 619423 MemQueueInit: repeated init [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.042 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.054 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=ms_scalar_summary. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.715.194 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.214 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.407 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.441 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.715.448 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.463 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.467 [tensor_data_transfer.cpp:898]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.493 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.501 [api_impl.cc:6077] 619423 MemQueueInit: Start to init queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.514 [npu_driver.cc:4076] 619423 MemQueueInit: init mem queue, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.529 [npu_driver.cc:4026] 619423 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.565 [npu_driver.cc:4087] 619423 MemQueueInit: repeated init [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.582 [api_impl.cc:6040] 619423 MemQueueCreate: Start to create queue on devId 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.594 [npu_driver.cc:3931] 619423 MemQueueCreate: create mem queue, deviceId=1, name=ms_histogram_summary. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.715.731 [queue_interface.c:182][ascend][curpid:619423,619423][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.751 [tensor_data_transfer.cpp:940]619423 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 1, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.869 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.715.901 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.715.924 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.924 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.715.946 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.716.073 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.716.107 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.127 [plugin_manager.cc:25] [619423]hcom ops plugin init start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.178 [plugin_manager.cc:38] [619423]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.201 [plugin_manager.cc:42] [619423]hcom running normal mode. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.248 [hcom_plugin.cc:50] [619423]Initialize start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.265 [hcom_plugin.cc:52] [619423]initializeCount_ : 1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.298 [hcom_plugin.cc:60] [619423]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.325 [hcom_plugin.cc:63] [619423]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.342 [hcom_plugin.cc:329] [619423]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.358 [hcom_plugin.cc:281] [619423]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:44.716.393 [hcom_plugin.cc:217] [619423]option profiling mode is not found. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.411 [hcom_plugin.cc:164] [619423][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.427 [hcom_plugin.cc:180] [619423][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.440 [hcom_plugin.cc:244] [619423]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.556 [hcom_plugin.cc:82] [619423]hccl ops plugin init success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.573 [plugin_manager.cc:48] [619423]hcom ops plugin init end. ret[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.587 [hcom_plugin.cc:126] [619423]get hccl kernel info store start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.612 [hcom_plugin.cc:132] [619423]get hccl kernel info store finished. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.716.643 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.716.669 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.684 [hcom_executor.cc:29] [619423]Hcom Excutor Initialize start. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.716.732 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.753 [adapter_rts.cc:234] [619423][hrtGetDevice]deviceLogicId[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.716.784 [api_c_soc.cc:70] 619423 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.822 [hcom_common.cc:85] [619423][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.853 [hcom_common.cc:88] [619423][HcomGetCurHcomCtx] is no Used deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.881 [hcom.cc:486] [619423]Get Hcom Init Status: [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.897 [hcom_common.cc:85] [619423][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.716.909 [hcom_rpc.cc:1019] [619423]Get Hcom Init Status: [0] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:44.716.935 [hcom_base_funs.cc:36] [619423][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:44.716.957 [hcom_executor.cc:121] [619423][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.016 [api_impl.cc:1120] 619423 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.043 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.103 [stream.cc:2246] 619423 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.164 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.182 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.672 [npu_driver.cc:6125] 619423 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.721 [npu_driver.cc:6137] 619423 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=412, streamId=4, logicCq=412, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.748 [npu_driver.cc:6189] 619423 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=4, logicCq=412. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.779 [npu_driver.cc:5916] 619423 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc5a1000, addr=dfffcc5a1000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.798 [stream.cc:653] 619423 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.717.864 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.717.884 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.222 [stream.cc:291] 619423 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.809 [stream.cc:688] 619423 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=1, streamResId=65535 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.718.831 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.718.844 [acl_resource_manager.cpp:816]619423 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.859 [api_impl.cc:1162] 619423 StreamCreate: Succ, flags=1536, stream_id=4, context=aaaaf7bf5f00 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.718.886 [stream.cc:73] [619423]Construct stream by stream type success, ptr[0xaaab02f1dfc0] ctx[0xaaaaf7bf5f00], stream id[4], cqId[4], logicCqid[412] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.925 [api_impl.cc:1120] 619423 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.951 [npu_driver.cc:404] 619423 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.718.982 [stream.cc:2246] 619423 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.018 [stream_sqcq_manage.cc:147] 619423 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.036 [npu_driver.cc:882] 619423 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.255 [npu_driver.cc:6125] 619423 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.306 [npu_driver.cc:6137] 619423 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=413, streamId=5, logicCq=413, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.329 [npu_driver.cc:6189] 619423 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=5, logicCq=413. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.347 [npu_driver.cc:5916] 619423 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc580000, addr=dfffcc580000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.364 [stream.cc:653] 619423 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619423,python3.9):2025-04-03-09:35:44.719.411 [devmm_svm.c:1972][ascend][curpid:619423,619423][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.430 [npu_driver.cc:6080] 619423 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.719.739 [stream.cc:291] 619423 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.378 [stream.cc:688] 619423 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=1, streamResId=65535 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.720.403 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.720.439 [acl_resource_manager.cpp:816]619423 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.455 [api_impl.cc:1162] 619423 StreamCreate: Succ, flags=1536, stream_id=5, context=aaaaf7bf5f00 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.720.474 [stream.cc:73] [619423]Construct stream by stream type success, ptr[0xaaab02f9a020] ctx[0xaaaaf7bf5f00], stream id[5], cqId[5], logicCqid[413] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:44.720.642 [sal.cc:619] [619667]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.720.673 [hcom_executor.cc:587] [619667]MessageProcessThreadLoop start queType:[0] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:44.720.690 [sal.cc:619] [619668]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.696 [api_impl.cc:2999] 619667 SetDevice: device_id=1, deviceMode=0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.720.718 [hcom_executor.cc:587] [619668]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.726 [device.cc:306] 619667 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.739 [runtime.cc:3393] 619667 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.739 [api_impl.cc:2999] 619668 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.755 [runtime.cc:3560] 619667 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.761 [device.cc:306] 619668 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.758 [api_impl.cc:2999] 619669 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.768 [api_impl.cc:2987] 619667 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.775 [runtime.cc:3393] 619668 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:44.720.774 [hcom_executor.cc:34] [619423][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.786 [device.cc:306] 619669 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.793 [runtime.cc:5239] 619667 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.800 [runtime.cc:3560] 619668 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.813 [api_impl.cc:2999] 619670 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.814 [runtime.cc:3393] 619669 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.825 [api_impl.cc:2987] 619668 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.826 [api_impl.cc:2992] 619667 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.720.827 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.837 [device.cc:306] 619670 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.843 [runtime.cc:3560] 619669 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.850 [runtime.cc:5239] 619668 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:44.720.862 [awatchdog_core.c:175](tid:619667) create watchdog for id : 65543, tid : 619667, timeout : 300s [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.720.871 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.870 [runtime.cc:3393] 619670 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.888 [api_impl.cc:2987] 619669 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.895 [api_impl.cc:2992] 619668 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.899 [api_impl.cc:3025] 619667 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.720.911 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.913 [runtime.cc:3560] 619670 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.919 [runtime.cc:5239] 619669 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:44.720.929 [awatchdog_core.c:175](tid:619668) create watchdog for id : 65543, tid : 619668, timeout : 300s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.941 [api_impl.cc:2987] 619670 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.720.949 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.950 [api_impl.cc:2992] 619669 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.953 [api_impl.cc:3025] 619668 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.958 [runtime.cc:5239] 619670 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.720.975 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:44.720.975 [awatchdog_core.c:175](tid:619669) create watchdog for id : 65543, tid : 619669, timeout : 300s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.985 [api_impl.cc:2992] 619670 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.720.993 [api_impl.cc:3025] 619669 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:44.721.002 [awatchdog_core.c:175](tid:619670) create watchdog for id : 65543, tid : 619670, timeout : 300s [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.010 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1132.25 msec. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.017 [api_impl.cc:3025] 619670 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.041 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [WARNING] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.075 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.129 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.014 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.183 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.220 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.244 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.091 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.448 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:44.721.476 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:44.721.567 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:44.721.604 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.721.657 [device.cpp:143]619671 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.689 [api_impl.cc:2999] 619671 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.706 [device.cc:306] 619671 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.720 [runtime.cc:3393] 619671 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.733 [runtime.cc:3560] 619671 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.750 [api_impl.cc:2987] 619671 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.768 [runtime.cc:5239] 619671 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.785 [api_impl.cc:2992] 619671 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:44.721.806 [awatchdog_core.c:175](tid:619671) create watchdog for id : 65543, tid : 619671, timeout : 300s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.721.822 [api_impl.cc:3025] 619671 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.721.837 [acl.cpp:115]619671 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.721.853 [acl.cpp:121]619671 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.721.869 [device.cpp:154]619671 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:44.721.920 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.249 msec. [WARNING] DEVICE(619423,fffebfffefa0,python3.9):2025-04-03-09:35:44.722.358 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 2 [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.868.195 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.868.195 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.868.225 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:44.868.235 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.242 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.251 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.257 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.267 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.868.277 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.868.281 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.868.293 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:44.868.298 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.309 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:44.868.314 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.036.690 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.036.727 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.036.744 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.036.761 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.036.785 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.036.858 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:45.222.568 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:45.222.618 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.664 msec. [WARNING] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:45.222.638 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:45.222.662 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.222.821 [device.cpp:143]619759 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.854 [api_impl.cc:2999] 619759 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.871 [device.cc:306] 619759 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.884 [runtime.cc:3393] 619759 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.901 [runtime.cc:3560] 619759 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.913 [api_impl.cc:2987] 619759 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.930 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.947 [api_impl.cc:2992] 619759 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:45.222.967 [awatchdog_core.c:175](tid:619759) create watchdog for id : 65543, tid : 619759, timeout : 300s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.222.984 [api_impl.cc:3025] 619759 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.222.997 [acl.cpp:115]619759 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.223.012 [acl.cpp:121]619759 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.223.025 [device.cpp:154]619759 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.308.310 [device.cpp:231]619759 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.360 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.522 [api_c_soc.cc:70] 619759 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.308.593 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.614 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.638 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.654 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.666 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.680 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=0. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.308.702 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=0.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.308.752 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.765 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.778 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.788 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.798 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.308.810 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=2. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.309.161 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=2.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.309.197 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.215 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.228 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.240 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.254 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.268 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=3. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.309.728 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=3.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.309.759 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.778 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.790 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.803 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.815 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.309.828 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=4. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.310.432 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=4.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.310.465 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.310.482 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.310.496 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.310.507 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.310.520 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.310.534 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=5. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.311.450 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=5.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.311.486 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.311.501 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.311.513 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.311.525 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.311.540 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.311.552 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=6. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.312.496 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=6.) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.312.534 [memory.cpp:559]619759 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.312.552 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.312.566 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.312.578 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.312.595 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.312.609 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=7. [INFO] DRV(619423,python3.9):2025-04-03-09:35:45.313.632 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=7.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.672 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.748 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.794 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.823 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.848 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.866 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.886 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.903 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.920 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.947 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.966 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.313.981 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.023 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.038 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.058 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.075 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.091 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.118 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.143 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.159 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.179 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.194 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.210 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.224 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.250 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.279 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.305 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.324 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.338 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.353 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.377 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.393 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.407 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.423 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.452 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.469 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.484 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.498 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.529 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.544 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.582 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.606 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.621 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.636 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.653 [runtime.cc:1664] 619759 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.666 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.689 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.709 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.726 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.762 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.788 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.800 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.816 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.830 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.851 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.874 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.886 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.900 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.916 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.933 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.956 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.968 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.984 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.314.999 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.014 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.040 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.066 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.079 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.093 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.108 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.134 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.149 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.163 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.178 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.194 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.221 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.235 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.248 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.264 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.279 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.299 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.314 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.328 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.754 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.806 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.849 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.865 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.881 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.898 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.920 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.961 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.978 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.315.992 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.014 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.036 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.069 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.083 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.095 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.111 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.133 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.200 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.224 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.241 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.260 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.276 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.316 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.332 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.344 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.361 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.385 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.424 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.439 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.453 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.467 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.485 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.526 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.543 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.555 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.571 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.600 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.643 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.658 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.670 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.685 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.701 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.742 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.758 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.772 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.788 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.812 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.851 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.868 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.882 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.898 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.917 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.956 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.968 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.980 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.316.997 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.022 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.058 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.071 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.084 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.098 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.113 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.152 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.174 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.190 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.204 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.227 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.273 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.288 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.302 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.315 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.330 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.367 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.381 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.393 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.407 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.432 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.470 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.486 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.500 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.514 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.531 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.578 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.593 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.605 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.618 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.634 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.674 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.689 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.702 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.718 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.731 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.775 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.793 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.806 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.819 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.837 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.882 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.895 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.910 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.924 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.947 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.986 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.317.998 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.009 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.023 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.039 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.077 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.092 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.104 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.158 [api_impl.cc:1990] 619759 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.465 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.318.498 [api_impl.cc:2664] 619759 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.365 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619419, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.407 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619427, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.446 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619431, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.466 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619435, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.484 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619439, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.498 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619443, num=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.517 [npu_driver.cc:3571] 619759 SetIpcMemPid: set ipc mem pid success, name=0009739f00000000000003160100ffc40001, pid[0]=619447, num=1 [2025-04-03 09:35:44.706854] [debug] [619423] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.706866] [debug] [619423] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.223149] [warn] [619759] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.223178] [debug] [619759] [lcal_sock_exchange.cpp:140] curRank: 1 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.223185] [debug] [619759] [lcal_sock_exchange.cpp:272] Client side 1 begin to connect [2025-04-03 09:35:45.223331] [debug] [619759] [lcal_sock_exchange.cpp:93] rank:1 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308246] [debug] [619759] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 1 rank :1 rankSize :8 [2025-04-03 09:35:45.308470] [debug] [619759] [lcal_comm.cpp:406] rank 1 devId: 1, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308484] [info] [619759] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308491] [info] [619759] [lcal_comm.cpp:242] rank 1/8 running devId:1 [2025-04-03 09:35:45.308540] [debug] [619759] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308583] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308743] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309187] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309752] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310456] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311476] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312524] [debug] [619759] [lcal_comm.cpp:340] 1 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313651] [debug] [619759] [lcal_comm.cpp:372] EnablePeerAccess succeed1 [2025-04-03 09:35:45.318121] [debug] [619759] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 1 [2025-04-03 09:35:45.318143] [debug] [619759] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.318486] [debug] [619759] [lcal_comm.cpp:455] peerMem[rank1], allocate finished. [2025-04-03 09:35:45.340205] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340247] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340256] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340263] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340270] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340277] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340283] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340290] [debug] [619759] [lcal_comm.cpp:472] rank : 1, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340296] [debug] [619759] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340529] [debug] [619759] [lcal_comm.cpp:557] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340763] [debug] [619759] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340772] [debug] [619759] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340777] [debug] [619759] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340782] [debug] [619759] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340796] [debug] [619759] [lcal_comm.cpp:515] rank[INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.340.853 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.341.999 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.342.018 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.342.682 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.342.699 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.343.198 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.343.217 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.357.933 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.357.950 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.359.071 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.359.090 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.359.957 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.359.973 [api_impl.cc:4286] 619759 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.547 [npu_driver.cc:3195] 619759 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.591 [api_impl.cc:1990] 619759 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.623 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.360.642 [memory.cpp:421]619759 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:45.360.832 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.360.886 [device.cpp:143]619759 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.924 [api_impl.cc:2999] 619759 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.940 [device.cc:306] 619759 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.953 [runtime.cc:3393] 619759 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.971 [runtime.cc:3560] 619759 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.983 [api_impl.cc:2987] 619759 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.360.995 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.361.012 [api_impl.cc:2992] 619759 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.361.032 [api_impl.cc:3025] 619759 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.361.046 [acl.cpp:115]619759 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.361.059 [acl.cpp:121]619759 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.361.078 [device.cpp:154]619759 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [WARNING] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:45.361.114 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.361.157 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.174 [adapter_rts.cc:198] [619759][hrtGetDeviceRefresh]deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.208 [externalinput.cc:2065] [619759]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.230 [externalinput.cc:490] [619759]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.242 [externalinput.cc:429] [619759]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.254 [externalinput.cc:543] [619759]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.269 [externalinput.cc:622] [619759]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.282 [externalinput.cc:714] [619759]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.295 [externalinput.cc:821] [619759]HCCL_IF_IP is not set [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.310 [externalinput.cc:877] [619759]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.320 [externalinput.cc:844] [619759]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.333 [externalinput.cc:728] [619759]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.345 [externalinput.cc:758] [619759]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.357 [externalinput.cc:806] [619759]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.369 [externalinput.cc:1508] [619759]HCCL_ALGO is not set [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.383 [externalinput.cc:1671] [619759][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.414 [externalinput.cc:462] [619759]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.427 [externalinput.cc:1519] [619759]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.439 [externalinput.cc:1554] [619759]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.361.591 [api_c_soc.cc:70] 619759 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.613 [externalinput.cc:1599] [619759]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.633 [externalinput.cc:1633] [619759]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.645 [externalinput.cc:1778] [619759]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.658 [externalinput.cc:1807] [619759]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.674 [externalinput.cc:515] [619759]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.689 [externalinput.cc:1841] [619759]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.701 [externalinput.cc:1929] [619759]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.713 [externalinput.cc:1949] [619759]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.725 [externalinput.cc:1969] [619759]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.744 [externalinput.cc:2005] [619759]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.758 [externalinput.cc:2036] [619759]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.770 [externalinput.cc:2209] [619759]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.784 [externalinput.cc:2178] [619759][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.797 [externalinput.cc:889] [619759]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.810 [externalinput.cc:914] [619759]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.822 [externalinput.cc:2240] [619759]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.835 [externalinput.cc:2308] [619759]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.846 [externalinput.cc:2339] [619759]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.857 [externalinput.cc:2296] [619759]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.869 [externalinput.cc:407] [619759]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:45.361.883 [externalinput.cc:672] [619759][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:45.361.915 [env_config.cc:127] [619759]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:45.361.929 [env_config.cc:131] [619759]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.953 [env_config.cc:302] [619759]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.965 [env_config.cc:302] [619759]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.361.979 [op_base.cc:1091] [619759]Entry-HcclCommInitRootInfoInner:ranks[8], rank[1], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.005 [op_base.cc:128] [619759][HcclGetDeviceId] deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.020 [op_base.cc:144] [619759][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.046 [rank_consistentcy_checker.cc:32] [619759][GetInstance] get deviceLogicId[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.362.083 [api_impl.cc:3100] 619759 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.362.098 [api_impl.cc:3111] 619759 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.362.110 [api_impl.cc:3130] 619759 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.320 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.352 [network_manager.cc:169] [619759]NetworkManager: devicePhyId[1], devicePhyId_[1] deviceLogicId_[1], hasBackup[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.394 [ra_host.c:1594]tid:619759,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.362.431 [adapter_hccp.cc:1450] [619759]set host socket whitelist status[0] success. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.464 [ra_host.c:205]tid:619759,ra_init(205) : Input parameters: phy_id[1], nic_position:[0] hdc_type:[6] [INFO] ROCE(619423,python3.9):2025-04-03-09:35:45.362.515 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.531 [ra_peer.c:776]tid:619759,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[1] start [INFO] ROCE(619423,python3.9):2025-04-03-09:35:45.362.598 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619423,python3.9):2025-04-03-09:35:45.362.670 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.700 [rs.c:327]tid:619759,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619423,python3.9):2025-04-03-09:35:45.362.865 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619423,python3.9):2025-04-03-09:35:45.362.892 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619423,python3.9):2025-04-03-09:35:45.362.906 [rs_ssl.c:1513]tid:619759,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.920 [rs_ssl.c:1529]tid:619759,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.362.963 [rs_epoll.c:692]tid:619759,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.151 [rs_epoll.c:713]tid:619759,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.167 [rs_epoll.c:623]tid:619775,rs_connect_handle(623) : thread begin! thread_id:281469582634912, pid:619423, ppid:619411 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.176 [rs_epoll.c:487]tid:619773,rs_epoll_handle(487) : thread begin! thread_id:281469591089056, pid:619423, ppid:619411 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.170 [rs.c:366]tid:619759,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.238000] ms [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.218 [rs_epoll.c:641]tid:619775,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.265 [rs_epoll.c:415]tid:619773,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.363.287 [rs_epoll.c:511]tid:619773,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.235 [rs.c:471]tid:619759,rs_init(471) : rs init success, chip_id[1] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.269 [ra_peer.c:813]tid:619759,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[1] succ [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.370.289 [adapter_hccp.cc:583] [619759]init ra success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.370.302 [network_manager.cc:222] [619759]NetworkManager nicDeploy[0] deviceLogicId_[1] devicePhyId_[1] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.332 [ra_host.c:1764]tid:619759,ra_get_ifnum(1764) : Input parameters: phy_id[1], nic_position:[0], is_all:[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.355 [ra_peer.c:859]tid:619759,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[1] start [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.384 [ra_peer.c:867]tid:619759,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[1] succ [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.370.401 [adapter_hccp.cc:1378] [619759][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.414 [ra_host.c:1814]tid:619759,ra_get_ifaddrs(1814) : Input parameters: phy_id[1], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.426 [ra_peer.c:874]tid:619759,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[1] start [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.439 [rs_socket.c:2743]tid:619759,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.452 [rs_socket.c:2743]tid:619759,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.464 [rs_socket.c:2743]tid:619759,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.476 [rs_socket.c:2749]tid:619759,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.488 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.498 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.514 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.529 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.542 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.556 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.566 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.576 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.588 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.600 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.612 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.637 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.650 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.663 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.677 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.689 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.699 [rs_socket.c:2749]tid:619759,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.710 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.721 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.734 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.747 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.756 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.767 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.779 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.789 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.801 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.813 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.823 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.835 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.845 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.857 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.873 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.888 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.898 [rs_socket.c:2749]tid:619759,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.910 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.921 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.933 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.945 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.960 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.977 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.370.989 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.008 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.021 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.035 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.045 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.054 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.065 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.077 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.088 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.102 [rs_socket.c:2751]tid:619759,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.113 [ra_peer.c:882]tid:619759,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[1] succ [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.132 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.148 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.163 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.176 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.190 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.205 [adapter_hccp.cc:1407] [619759][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.224 [sal.cc:365] [619759]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.238 [sal.cc:432] [619759]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.257 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:45.371.295 [ra_host.c:340]tid:619759,ra_socket_init_v1(340) : socket init:mode=0 phy_id=1 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.309 [adapter_hccp.cc:818] [619759]socket init v1 success, socketHandle[0xfffeac0faba0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.323 [network_manager.cc:1218] [619759]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.333 [network_manager.cc:1257] [619759]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffeac0faba0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.383 [adapter_hal.cc:298] [619759]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.406 [sal.cc:593] [619759]IsHostUseDevNic[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.418 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.431 [network_manager.cc:169] [619759]NetworkManager: devicePhyId[4294967295], devicePhyId_[1] deviceLogicId_[1], hasBackup[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.476 [tsd_client.cpp:189][TsdCapabilityGet][tid:619759] TsdCapabilityGet Begin. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.497 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.511 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.523 [process_mode_manager.cpp:1042][CapabilityGet][tid:619759] [ProcessModeManager] enter into CapabilityGet process deviceId[1] type[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.535 [adapter_tdt.cc:66] [619759]Get TsdCapability success. deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.547 [network_manager.cc:175] [619759][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:45.371.565 [network_manager.cc:99] [619759][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.583 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.595 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.607 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619759] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.619 [process_mode_manager.cpp:386][InitTsdClient][tid:619759] [TsdClient] tsd client has been inited before [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.639 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619759] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619423,python3.9):2025-04-03-09:35:45.371.664 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619759] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.677 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619759] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.690 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619759] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.704 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619759] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.727 [version_verify.cpp:114][SpecialFeatureCheck][tid:619759] VersionVerify: previous type[48], supported [INFO] TDT(619423,python3.9):2025-04-03-09:35:45.371.764 [hdc_client.cpp:154][TsdRecvData][tid:619759] recv timeout is:150000 ms [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.380.177 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.380.184 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.380.208 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.380.248 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.254 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.263 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.270 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.280 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.380.288 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.380.296 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.380.301 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.380.312 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.318 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.380.326 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.181 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.182 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.181 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.182 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.182 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.540.185 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.301 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.319 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.332 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.340 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.348 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.356 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.378 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.378 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.391 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.383 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.400 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.409 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.412 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.416 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.422 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.429 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.435 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.441 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.451 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.456 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.465 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.468 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.473 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.482 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.486 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.493 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.499 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.504 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.515 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.522 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.535 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.541 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.540.566 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.572 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.581 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.540.594 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.892.181 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.892.181 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.892.211 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:45.892.220 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.227 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.238 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.244 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.251 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.892.262 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.892.265 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.892.276 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:45.892.283 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.291 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:45.892.298 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.252.706 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:46.252.734 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:46.252.752 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:46.252.766 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.168 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.168 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.170 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.170 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.177 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.372.177 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.271 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.288 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.298 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.305 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.317 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.321 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.329 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.331 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.332 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.342 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.354 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.358 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.362 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.363 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.369 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.382 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.388 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.393 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.398 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.403 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.412 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.415 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.434 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.438 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.445 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.451 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.459 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.464 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.476 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.483 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.487 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.372.499 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.506 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.512 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.522 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.372.527 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.404.192 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.404.193 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.404.222 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.404.232 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.238 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.247 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.252 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.260 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.404.266 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.404.274 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.404.295 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.404.300 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.310 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.404.315 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.916.177 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.916.177 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.916.206 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:46.916.215 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.224 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.233 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.239 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.247 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.916.254 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.916.261 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.916.270 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:46.916.273 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.286 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:46.916.289 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.170 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.172 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.173 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.174 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.175 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.204.180 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.276 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.291 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.303 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.306 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.320 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.323 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.333 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.334 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.340 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.349 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.357 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.363 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.365 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.369 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.376 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.389 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.393 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.398 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.406 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.407 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.410 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.419 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.431 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.435 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.434 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.441 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.447 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.472 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.476 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.481 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.487 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.500 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.204.504 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.513 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.523 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.204.537 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.428.173 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.428.173 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.428.200 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.428.208 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.218 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.227 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.233 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.242 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.428.251 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.428.256 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.428.264 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.428.268 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.281 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.428.284 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.730.775 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.730.794 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619759] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.730.823 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619759] [TsdClient] DeviceMsgProc recvMsg realDeviceId[1] msgType[49] localDevId[1] rspCode[0]heterogeneousSubPid[31560], tsdSupportLevel_[63] [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.730.844 [hdc_client.cpp:172][TsdRecvData][tid:619759] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.730.861 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619759] OpenSubProc success stype:0, pid:31560 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.730.874 [adapter_tdt.cc:39] [619759]Open TsdProcess success. deviceLogicId[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.730.871 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.730.885 [network_manager.cc:118] [619759][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31560], devicePhyId_[1], deviceLogicId_[1], hasBackup[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.730.899 [network_manager.cc:180] [619759][NetworkManager][Init]open tsd by process success, devicePhyId[1], deviceLogicId_[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.730.971 [ra_host.c:205]tid:619759,ra_init(205) : Input parameters: phy_id[1], nic_position:[1] hdc_type:[18] [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.730.990 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619423,python3.9):2025-04-03-09:35:47.731.015 [devdrv_hccl.c:103][ascend][curpid:619423,619759][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619423) [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.731.035 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.731.053 [ra_hdc.c:580]tid:619759,ra_hdc_init(580) : hdc init start! logic id is 1, phy id is 1, hdc_type is 18 [INFO] DRV(619423,python3.9):2025-04-03-09:35:47.731.289 [hdc_core.c:1982][ascend][curpid:619423,619759][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4263; pid=619423) [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.731.521 [ra_hdc.c:616]tid:619759,ra_hdc_init(616) : hdc init OK! phy_id[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.131 [adapter_hccp.cc:583] [619759]init ra success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.149 [network_manager.cc:222] [619759]NetworkManager nicDeploy[1] deviceLogicId_[1] devicePhyId_[1] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.184 [topoinfo_detect.cc:534] [619759]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.735.210 [ra_host.c:1764]tid:619759,ra_get_ifnum(1764) : Input parameters: phy_id[1], nic_position:[1], is_all:[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.596 [adapter_hccp.cc:1538] [619759][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.735.616 [ra_host.c:1814]tid:619759,ra_get_ifaddrs(1814) : Input parameters: phy_id[1], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.955 [adapter_hccp.cc:1568] [619759][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth1] ip[29.105.19.202%eth1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.735.976 [adapter_hccp.cc:1568] [619759][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth1] ip[fe80::2287:ecff:fe02:45ec%eth1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.005 [topoinfo_detect.cc:577] [619759]select AF_INET family as device socket family. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.027 [topoinfo_detect.cc:756] [619759]no backup device ip: use 0 as device ip. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.040 [topoinfo_detect.cc:294] [619759][HCCL_TRACE]SetupAgent rankNum[8], rank[1], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[1], phydevId[1], deviceIp[29.105.19.202] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.080 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.099 [hccl_socket.cc:516] [619759][Get][NicHandleInfo]phyId[1], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.118 [hccl_socket.cc:217] [619759][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281469273484192], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.736.134 [adapter_hccp.cc:1050] [619759]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.153 [ra_host.c:759]tid:619759,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[1], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.201 [rs_socket.c:1810]tid:619759,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.232 [rs_socket.c:1819]tid:619759,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619423,python3.9):2025-04-03-09:35:47.736.252 [rs_socket.c:132]tid:619759,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.273 [rs_socket.c:1880]tid:619759,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.728 [rs_socket.c:1577]tid:619775,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:48913, fd:53 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.767 [rs_drv_socket.c:59]tid:619775,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:53 begin [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.871 [rs_drv_socket.c:98]tid:619775,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:48913, server addr 192.168.0.44:60008, fd:53 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.896 [rs_socket.c:1427]tid:619775,rs_conn_cost_time(1427) : socket [53] connect success! cost [0.624000] ms [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.736.955 [rs_socket.c:1392]tid:619775,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.737.382 [topoinfo_exchange_agent.cc:212] [619759]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.737.450 [topoinfo_exchange_agent.cc:236] [619759]local rank[1] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.737.471 [topoinfo_exchange_agent.cc:43] [619759]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.737.514 [topoinfo_exchange_agent.cc:345] [619759]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.737.625 [topoinfo_exchange_agent.cc:77] [619759]topo exchange client send rank basic info success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.800.805 [api_impl.cc:2096] 619759 HostMalloc: size=2485 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.800.837 [api_error.cc:1260] 619759 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.801.109 [api_error.cc:1278] 619759 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.131 [topoinfo_exchange_agent.cc:80] [619759]topo exchange client get rank basic info success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.185 [topoinfo_detect.cc:303] [619759]topo detect completed. myrank[1], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.219 [config.cc:198] [619759]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.241 [config.cc:210] [619759]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.255 [config.cc:210] [619759]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.269 [config.cc:210] [619759]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.283 [config.cc:210] [619759]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.294 [config.cc:210] [619759]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.304 [config.cc:210] [619759]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.317 [config.cc:210] [619759]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.328 [config.cc:210] [619759]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.801.342 [adapter_hccp.cc:1067] [619759]ra socket batch close fdhandle[0xfffeac054bf0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.801.358 [ra_host.c:796]tid:619759,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[1], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.801.376 [rs_socket.c:1937]tid:619759,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:53, state:9 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.848.208 [adapter_hccp.cc:1099] [619759]ra socket batch close success,take time [46851]us [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:47.848.242 [hccl_socket.cc:232] [619759][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.848.263 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.284 [ra_host.c:404]tid:619759,ra_socket_deinit(404) : Input parameters: phy_id[1] family[2] local_ip[192.168.0.44] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.300 [rs.c:1629]tid:619759,rs_socket_deinit(1629) : rs socket deinit start, phy_id:1 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.324 [rs.c:1641]tid:619759,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.354 [rs.c:1653]tid:619759,rs_socket_deinit(1653) : socket deinit success, phy_id:1, local_ip:192.168.0.44 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.848.367 [network_manager.cc:1233] [619759]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.848.381 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.394 [ra_host.c:248]tid:619759,ra_deinit(248) : Input parameters: phy_id[1], nic_position:[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.406 [ra_peer.c:821]tid:619759,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[1] start [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.424 [rs.c:1783]tid:619759,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619423,python3.9):2025-04-03-09:35:47.848.428 [rs_epoll.c:523]tid:619773,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:51 [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.848.445 [rs_epoll.c:544]tid:619773,rs_epoll_handle(544) : QUIT thread_id:281469591089056, pid:619423, read num:8 [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.940.171 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.940.171 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.940.195 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:47.940.202 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.213 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.220 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.227 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.232 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.940.240 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.940.247 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.940.251 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:47.940.263 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.269 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:47.940.277 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.947.410 [rs_epoll.c:679]tid:619775,rs_connect_handle(679) : QUIT thread_id:281469582634912, pid:619423 [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.948.723 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.948.747 [rs.c:1810]tid:619759,rs_deinit(1810) : rs_deinit chip_id[1] ok [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.948.762 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.948.786 [ra_peer.c:852]tid:619759,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[1] succ [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.948.798 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.948.833 [network_manager.cc:409] [619759]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.948.850 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.948.862 [ra_host.c:248]tid:619759,ra_deinit(248) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.948.873 [ra_hdc.c:651]tid:619759,ra_hdc_deinit(651) : hdc deinit start! phy_id[1] [INFO] DRV(619423,python3.9):2025-04-03-09:35:47.949.055 [hdc_client.c:417][ascend][curpid:619423,619759][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4263) [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.949.129 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:47.949.149 [ra_hdc.c:685]tid:619759,ra_hdc_deinit(685) : hdc deinit OK! phy_id[1] [INFO] ROCE(619423,python3.9):2025-04-03-09:35:47.949.167 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.234 [tsd_client.cpp:189][TsdCapabilityGet][tid:619759] TsdCapabilityGet Begin. [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.255 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.269 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.282 [process_mode_manager.cpp:1042][CapabilityGet][tid:619759] [ProcessModeManager] enter into CapabilityGet process deviceId[1] type[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:47.949.295 [adapter_tdt.cc:66] [619759]Get TsdCapability success. deviceLogicId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.312 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.322 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.331 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619759] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.346 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619759] add close subproc:31560, proctype:0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.361 [version_verify.cpp:114][SpecialFeatureCheck][tid:619759] VersionVerify: previous type[60], supported [INFO] TDT(619423,python3.9):2025-04-03-09:35:47.949.403 [hdc_client.cpp:154][TsdRecvData][tid:619759] recv timeout is:150000 ms [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.177 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.178 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.180 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.180 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.182 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.036.205 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.284 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.304 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.312 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.322 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.329 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.335 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.341 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.344 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.347 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.352 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.358 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.362 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.369 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.374 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.383 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.386 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.396 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.408 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.413 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.427 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.439 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.445 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.453 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.458 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.488 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.493 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.478 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.506 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.513 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.558 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.571 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.571 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.631 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.036.661 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.694 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.036.712 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.201.480 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619759] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.201.510 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619759] [TsdClient] DeviceMsgProc recvMsg realDeviceId[1] msgType[61] localDevId[1] rspCode[0]heterogeneousSubPid[31560], tsdSupportLevel_[63] [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.201.526 [hdc_client.cpp:172][TsdRecvData][tid:619759] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.201.539 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619759] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.201.552 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619759] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.563 [adapter_tdt.cc:52] [619759]Open TsdProcess success. deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.579 [network_manager.cc:405] [619759][DeInit]finish CloseHccpProcess, phy_id[1], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.592 [network_manager.cc:409] [619759]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.606 [topoinfo_detect.cc:555] [619759]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.618 [topoinfo_detect.cc:321] [619759]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.632 [topoinfo_detect.cc:219] [619759]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.653 [rank_consistentcy_checker.cc:32] [619759][GetInstance] get deviceLogicId[1] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:48.201.685 [rank_consistentcy_checker.cc:144] [619759][Record][CannVersion] version information str is empty. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.697 [op_base.cc:2231] [619759]rank table is null, rankTableCrc is 0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.764 [op_base.cc:2219] [619759]params.id.internal [hccl-619423-0-1743644148-hccl_world_group] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.789 [op_base.cc:993] [619759]rootInfo[hccl-619423-0-1743644148-hccl_world_group], params.logiceDevice[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.814 [hccl_comm.cc:98] [619759]hcclComm init workmode [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.201.829 [hccl_comm.cc:954] [619759]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.202.734 [hccl_comm.cc:1365] [619759]RegistTaskAbortHandler begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.202.757 [task_abort_handler.cc:126] [619759]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.202.773 [api_impl.cc:4192] 619759 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.202.792 [runtime.cc:4698] 619759 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.202.845 [adapter_rts.cc:42] [619759]open so: libruntime.so [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.127 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.166 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.183 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.204 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.219 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.236 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.252 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.272 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.287 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.307 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.319 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.338 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.352 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.369 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.381 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.398 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.427 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.443 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.454 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.471 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.485 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.503 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.515 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.535 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.548 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.565 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.578 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.596 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.609 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.627 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.641 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.656 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.669 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.686 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.699 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.716 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.727 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.743 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.755 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.775 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.786 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.812 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.828 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.845 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.860 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.880 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.892 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.910 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.921 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.938 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.952 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.203.970 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.203.983 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.000 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.016 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.034 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.086 [hccl_communicator_attrs.cc:407] [619759]module[0] contains [8]devices [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.102 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.117 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.127 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.137 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.150 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.181 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.194 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.206 [hccl_communicator_attrs.cc:411] [619759]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.227 [hccl_communicator_attrs.cc:425] [619759]different module contains different numbers of cards:[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.241 [hccl_communicator_attrs.cc:426] [619759]different module contains different type of cards:[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.302 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.328 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.346 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.367 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.386 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.402 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.414 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.441 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.454 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.472 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.484 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.504 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.516 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.532 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.543 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.561 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.576 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.593 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.605 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.623 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.638 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.654 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.666 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.691 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.705 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.724 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.735 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.754 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.770 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.791 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.804 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.821 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.835 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.858 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.872 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.889 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.903 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.918 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.933 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.951 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.204.968 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.204.988 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.003 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.021 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.034 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.050 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.063 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.093 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.111 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.131 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.149 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.170 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.183 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.203 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.218 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.234 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.246 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.263 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.278 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.295 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.308 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.325 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.341 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.357 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.371 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.390 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.404 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.423 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.437 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.459 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.474 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.495 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.522 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.540 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.553 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.569 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.584 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.600 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.614 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.633 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.647 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.667 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.679 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.698 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.711 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.731 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.745 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.763 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.777 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.796 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.810 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.828 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.840 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.855 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.868 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.886 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.899 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.930 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.944 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.205.966 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.205.979 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.000 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.013 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.030 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.043 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.062 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.076 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.093 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.105 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.127 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.142 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.159 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.174 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.193 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.205 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.224 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.236 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.255 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.266 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.284 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.295 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.316 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.346 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.364 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.376 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.393 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.406 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.425 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.439 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.457 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.468 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.485 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.499 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.514 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.529 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.545 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.558 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.574 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.586 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.604 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.614 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.632 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.647 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.664 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.677 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.693 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.718 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.737 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.748 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.765 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.776 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.795 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.807 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.824 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.836 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.854 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.867 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.887 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.898 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.914 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.927 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.945 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.959 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.206.977 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.206.991 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.011 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.022 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.039 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.072 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.093 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.117 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.134 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.152 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.170 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.184 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.200 [hccl_communicator_attrs.cc:156] [619759][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.220 [hccl_communicator_attrs.cc:1021] [619759][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.236 [hccl_communicator.cc:1440] [619759][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.247 [hccl_communicator.cc:1450] [619759][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.260 [externalinput.cc:1445] [619759]InitParsesProtocol start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.273 [externalinput.cc:1462] [619759]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:48.207.287 [externalinput.cc:1411] [619759]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.306 [externalinput.cc:1413] [619759]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.319 [externalinput.cc:1467] [619759]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.344 [config.cc:557] [619759][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.367 [hccl_communicator.cc:668] [619759][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.390 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.406 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.421 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=0. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.442 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=0.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.466 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.481 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.498 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.513 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=2. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.528 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=2.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.541 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.554 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.571 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.585 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=3. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.601 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=3.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.612 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.626 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.641 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.657 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=4. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.672 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=4.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.688 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.699 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.712 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.726 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=5. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.741 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=5.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.754 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.765 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.779 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.794 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=6. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.809 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=6.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.825 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.836 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.848 [api_impl.cc:3520] 619759 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.207.860 [api_impl.cc:3168] 619759 EnableP2P: Enable P2P devIdDes=1, phyIdSrc=7. [INFO] DRV(619423,python3.9):2025-04-03-09:35:48.207.882 [devdrv_pcie.c:888][ascend][curpid:619423,619759][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=1; peer_phy_devid=7.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.897 [p2p_mgmt.cc:74] [619759]enable p2p: local logic id:1, remote physic id:7. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.920 [adapter_hal.cc:298] [619759]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.939 [sal.cc:593] [619759]IsHostUseDevNic[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.207.981 [adapter_hal.cc:298] [619759]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.000 [sal.cc:593] [619759]IsHostUseDevNic[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.013 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.024 [network_manager.cc:169] [619759]NetworkManager: devicePhyId[4294967295], devicePhyId_[1] deviceLogicId_[1], hasBackup[0] [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.041 [tsd_client.cpp:189][TsdCapabilityGet][tid:619759] TsdCapabilityGet Begin. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.051 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.062 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.075 [process_mode_manager.cpp:1042][CapabilityGet][tid:619759] [ProcessModeManager] enter into CapabilityGet process deviceId[1] type[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.088 [adapter_tdt.cc:66] [619759]Get TsdCapability success. deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.101 [network_manager.cc:175] [619759][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:48.208.117 [network_manager.cc:99] [619759][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.131 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.145 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619759] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.165 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619759] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.186 [process_mode_manager.cpp:386][InitTsdClient][tid:619759] [TsdClient] tsd client has been inited before [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.197 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619759] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619423,python3.9):2025-04-03-09:35:48.208.214 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619759] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.226 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619759] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.241 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619759] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.259 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619759] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.208.306 [hdc_client.cpp:154][TsdRecvData][tid:619759] recv timeout is:150000 ms [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.452.167 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.452.180 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.452.196 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.452.203 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.211 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.220 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.226 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.234 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.452.241 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.452.247 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.452.254 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.452.259 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.270 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.452.276 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.165 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.168 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.169 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.173 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.177 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.868.182 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.288 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.309 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.318 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.323 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.332 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.337 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.348 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.354 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.355 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.362 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.368 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.375 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.380 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.388 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.392 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.393 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.401 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.412 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.422 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.428 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.434 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.441 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.444 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.457 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.463 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.471 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.494 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.498 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.513 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.514 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.553 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.560 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.603 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.868.638 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.677 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.868.691 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.940.704 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.940.732 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.940.751 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:48.940.766 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.964.176 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.964.176 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.964.204 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:48.964.212 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.221 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.228 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.236 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.241 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.964.250 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.964.255 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.964.265 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:48.964.287 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.294 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:48.964.300 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.476.172 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.476.174 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.476.197 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.476.204 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.214 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.222 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.226 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.234 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.476.240 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.476.246 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.476.253 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.476.259 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.268 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.476.274 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.169 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.171 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.172 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.173 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.178 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.700.180 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.271 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.287 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.299 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.311 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.312 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.326 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.330 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.343 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.346 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.346 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.388 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.356 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.403 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.387 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.408 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.417 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.431 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.443 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.449 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.457 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.465 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.476 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.483 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.435 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.466 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.490 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.497 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.507 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.531 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.537 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.700.544 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.575 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.588 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.595 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.618 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.700.604 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.988.183 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.988.183 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.988.214 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:49.988.223 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.231 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.241 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.246 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.256 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.988.263 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.988.268 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.988.278 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:49.988.283 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.292 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:49.988.303 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] TDT(619423,python3.9):2025-04-03-09:35:50.312.453 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619759] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.312.457 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:50.312.476 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619759] [TsdClient] DeviceMsgProc recvMsg realDeviceId[1] msgType[49] localDevId[1] rspCode[0]heterogeneousSubPid[31579], tsdSupportLevel_[63] [INFO] TDT(619423,python3.9):2025-04-03-09:35:50.312.495 [hdc_client.cpp:172][TsdRecvData][tid:619759] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:50.312.507 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619759] OpenSubProc success stype:0, pid:31579 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.312.520 [adapter_tdt.cc:39] [619759]Open TsdProcess success. deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.312.533 [network_manager.cc:118] [619759][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31579], devicePhyId_[1], deviceLogicId_[1], hasBackup[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.312.536 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.312.547 [network_manager.cc:180] [619759][NetworkManager][Init]open tsd by process success, devicePhyId[1], deviceLogicId_[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:50.312.592 [ra_host.c:205]tid:619759,ra_init(205) : Input parameters: phy_id[1], nic_position:[1] hdc_type:[18] [INFO] ROCE(619423,python3.9):2025-04-03-09:35:50.312.668 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619423,python3.9):2025-04-03-09:35:50.312.687 [devdrv_hccl.c:103][ascend][curpid:619423,619759][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619423) [INFO] ROCE(619423,python3.9):2025-04-03-09:35:50.312.725 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:50.312.739 [ra_hdc.c:580]tid:619759,ra_hdc_init(580) : hdc init start! logic id is 1, phy id is 1, hdc_type is 18 [INFO] DRV(619423,python3.9):2025-04-03-09:35:50.312.964 [hdc_core.c:1982][ascend][curpid:619423,619759][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4277; pid=619423) [INFO] HCCP(619423,python3.9):2025-04-03-09:35:50.313.181 [ra_hdc.c:616]tid:619759,ra_hdc_init(616) : hdc init OK! phy_id[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.583 [adapter_hccp.cc:583] [619759]init ra success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.608 [network_manager.cc:222] [619759]NetworkManager nicDeploy[1] deviceLogicId_[1] devicePhyId_[1] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.627 [hccl_communicator_attrs.cc:1307] [619759][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.638 [hccl_communicator_attrs.cc:1275] [619759]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.657 [hccl_communicator.cc:2259] [619759][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.682 [adapter_hccp.cc:1650] [619759]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.857 [adapter_hccp.cc:2343] [619759]Get available Vnic info success, deviceIdType[0] deviceId[0x1] Vnic ip[192.168.2.198] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.316.984 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:50.317.025 [ra_host.c:280]tid:619759,ra_socket_init(280) : socket init:mode=1 phy_id=1 family=2 ip=192.168.2.198 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.072 [adapter_hccp.cc:806] [619759]socket init success, ip[3322063040] device id[1], socketHandle[0xfffeac0ffbb0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.087 [network_manager.cc:692] [619759]devicePhyId[1], ip[192.168.2.198] socket init OK [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.099 [network_manager.cc:432] [619759][Start][Vnic]trying to listen on ip[192.168.2.198] port[16666]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:50.317.120 [ra_host.c:875]tid:619759,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[1], local_ip[192.168.2.198], port[16666] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.518 [network_manager.cc:448] [619759][Start][Vnic]Listen on ip[192.168.2.198], port[16666] sucess, devPhyId[1], devLogicId[1], isAutoPort[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.536 [network_manager.cc:451] [619759]NetworkManager devicePhyId_[1] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.551 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.564 [hccl_socket.cc:106] [619759][HcclSocket][Listen] device[1] listens on ip[192.168.2.198] port[16666] success, socketType[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.581 [hccl_socket_manager.cc:56] [619759][Init][Server]ip[192.168.2.198] port[16666] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.603 [hccl_communicator.cc:1548] [619759][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.198] port[16666] success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.618 [hccl_communicator_attrs.cc:1307] [619759][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.630 [hccl_communicator_attrs.cc:1275] [619759]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.649 [hccl_communicator.cc:1565] [619759]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.663 [network_manager.cc:69] [619759]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.677 [hccl_communicator.cc:2028] [619759]profiling config information:options[], mode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.317.905 [adapter_trace.cc:21] [619759]Call TraceCreate, return value[0] [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.317.932 [atrace_api.c:61](tid:619759) AtraceCreateWithAttr start. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.318.024 [trace_rb_log.c:45](tid:619759) init ring buffer time finished, realTime 1743644150318018299, monotonicTime 4211952795090, freq 99995 kHz, minutesWest 480 minutes [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.318.042 [trace_rb_log.c:137](tid:619759) [HCCL_619759_1] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.318.061 [tracer_mgr_operate.c:66](tid:619759) create object HCCL_619759_1 success, exitSave(true), noLock(false). [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.318.073 [atrace_api.c:63](tid:619759) AtraceCreateWithAttr end. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.085 [adapter_trace.cc:70] [619759]Call TraceCreateWithAttr, return value[0xaaaae6f988f8], Params: objName[HCCL_619759_1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.145 [rank_consistentcy_checker.cc:32] [619759][GetInstance] get deviceLogicId[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.193 [api_impl.cc:1486] 619759 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.218 [npu_driver.cc:4604] 619759 GetMaxStreamAndTask: deviceId=1, tsId=0, streamCount=1984. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.234 [task_exception_handler.cc:1212] [619759]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.266 [hccl_communicator_attrs.cc:1099] [619759][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.319 [api_impl.cc:4219] 619759 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.340 [command_handle.cc:25] [619759][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.353 [command_handle.cc:99] [619759][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.455 [profiler_manager_impl.cc:76] [619759][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.474 [hccl_communicator.cc:2193] [619759][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.318.487 [hccl_communicator.cc:803] [619759]Will use ffts mode. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.513 [api_impl.cc:3138] 619759 GetDeviceIndexByPhyId: get Index by PhyId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.531 [api_impl.cc:3149] 619759 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.546 [api_impl.cc:3157] 619759 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.318.558 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.018 [api_impl.cc:3476] 619759 GetDeviceInfo: get device info, deviceId=1 [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:50.320.070 [tbe_vector_reduce.cc:508] [619759]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.618 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.791 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.320.814 [dispatcher.cc:157] [619759][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.834 [api_impl.cc:3138] 619759 GetDeviceIndexByPhyId: get Index by PhyId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.850 [api_impl.cc:3149] 619759 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.863 [api_impl.cc:3157] 619759 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.875 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.320.902 [api_impl.cc:1990] 619759 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.321.747 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.321.803 [api_impl.cc:1990] 619759 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.321.827 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.321.863 [api_impl.cc:1990] 619759 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.321.878 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.321.908 [opexecounter.cc:71] [619759]alloc counter mem resource. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.321.962 [hccl_communicator_attrs.cc:1368] [619759][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.007 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.025 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.047 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.060 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.073 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.084 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.098 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.109 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.122 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.137 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.152 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.163 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.176 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.189 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.203 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.214 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.225 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.238 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.250 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.262 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.276 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.293 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.318 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.332 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.347 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.363 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.373 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.384 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.396 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.413 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.425 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.436 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.448 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.463 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.476 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.489 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.502 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.514 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.527 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.543 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.556 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.569 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.580 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.594 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.606 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.629 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.644 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.656 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.669 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.680 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.695 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.708 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.722 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.733 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.746 [alg_configurator.cc:68] [619759][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.758 [alg_configurator.cc:241] [619759][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.769 [alg_configurator.cc:134] [619759]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.790 [alg_configurator.cc:449] [619759][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.805 [alg_configurator.cc:42] [619759]alg configurator init success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.831 [topo_info_extractor.cc:66] [619759]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[1], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.852 [topo_info_extractor.cc:155] [619759][SetRankInfo]rankData_: userRank[1], devicePhyId[1], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.322.876 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.904 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.322.940 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.958 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.322.973 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.322.993 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.039 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.061 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.077 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.093 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.109 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.125 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.140 [api_error.cc:3478] 619759 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.157 [adapter_rts.cc:940] [619759][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.174 [topo_info_extractor.cc:348] [619759]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.315 [topo_info_extractor.cc:888] [619759]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[1], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.344 [topo_info_extractor.cc:585] [619759][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.366 [topo_info_extractor.cc:646] [619759][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.386 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.403 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/1; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.420 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.436 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.453 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.469 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.487 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.501 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.514 [topo_info_extractor.cc:689] [619759]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[1], userRankSize[8], plane size[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.535 [topo_info_extractor.cc:697] [619759][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.550 [topo_info_extractor.cc:734] [619759]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 1/1/0/; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.567 [topo_info_extractor.cc:739] [619759]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[1], userRankSize[8], plane size[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.581 [topo_info_extractor.cc:585] [619759][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.598 [topo_info_extractor.cc:646] [619759][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.616 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.639 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/1; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.658 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.674 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.690 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.705 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.722 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.741 [topo_info_extractor.cc:685] [619759]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.753 [topo_info_extractor.cc:689] [619759]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[1], userRankSize[8], plane size[8] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.798 [topo_info_extractor.cc:790] [619759]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId1/1; ] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.323.817 [topo_info_extractor.cc:519] [619759]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.946 [api_impl.cc:1990] 619759 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.323.974 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.324.021 [hccl_impl.cc:59] [619759]start hccl resources build:get deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.324.181 [hccl_impl.cc:177] [619759]hcclImpl init success. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:50.324.217 [hccl_aiv.cc:279] [619759][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.326.938 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.013 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.138 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.159 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.179 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.199 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.221 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.320 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.338 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.353 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.369 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.388 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.495 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.512 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.526 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.540 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.561 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.657 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.674 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.689 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.704 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.723 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.818 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.834 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.848 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.862 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.885 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.327.988 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.005 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.018 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.034 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.051 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.147 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.178 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.201 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.215 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.236 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.329 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.345 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.361 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.375 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.393 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.486 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.502 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.518 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.533 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.555 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.649 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.665 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.680 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.695 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.713 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.809 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.825 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.847 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.862 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.881 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.975 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.328.992 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.005 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.018 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.035 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.127 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.144 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.157 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.172 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.192 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.288 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.306 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.321 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.334 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.354 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.450 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.466 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.479 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.495 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.515 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.608 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.624 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.640 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.663 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.685 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.779 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.795 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.811 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.826 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.849 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.947 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.965 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.983 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.329.998 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.019 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.117 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.134 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.150 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.165 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.185 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.279 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.295 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.309 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.323 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.346 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.444 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.461 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.477 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.493 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.524 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.620 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.637 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.654 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.667 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.687 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.780 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.798 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.814 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.829 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.849 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.941 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.959 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.975 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.330.989 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.008 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.101 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.120 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.134 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.148 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.166 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.264 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.281 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.296 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.309 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.331 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.423 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.450 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.465 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.478 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.496 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.591 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.610 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.624 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.638 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.658 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.751 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.770 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.785 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.799 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.818 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.915 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.933 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.949 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.963 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.331.980 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.078 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.095 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.109 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.123 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.141 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.244 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.266 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.288 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.303 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.324 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.422 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.438 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.452 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.466 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.486 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.591 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.607 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.621 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.637 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.657 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.749 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.766 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.782 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.796 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.816 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.908 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.925 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.941 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.956 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.332.976 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.066 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.082 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.096 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.121 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.141 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.241 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.258 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.275 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.291 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.312 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.407 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.424 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.438 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.452 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.471 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.565 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.583 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.598 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.612 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.633 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.727 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.744 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.761 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.775 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.795 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.899 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.916 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.931 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.947 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.333.968 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.069 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.085 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.101 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.115 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.134 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.229 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.249 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.264 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.278 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.297 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.393 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.410 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.425 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.439 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.456 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.553 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.571 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.586 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.600 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.617 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.711 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.728 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.744 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.756 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.774 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.872 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.898 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.914 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.928 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.334.947 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.047 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.064 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.078 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.091 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.110 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.211 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.229 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.244 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.260 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.282 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.380 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.396 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.412 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.429 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.449 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.549 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.568 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.584 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.598 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.620 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.718 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.734 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.758 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.775 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.793 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.882 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.899 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.914 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.927 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.335.946 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.039 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.057 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.071 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.085 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.104 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.218 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.244 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.261 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.278 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.296 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.392 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.409 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.423 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.437 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.455 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.555 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.574 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.589 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.604 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.648 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.738 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.757 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.772 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.785 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.804 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.897 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.913 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.930 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.943 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.336.961 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.057 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.074 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.090 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.105 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.126 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.226 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.244 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.260 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.273 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.294 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.389 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.408 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.424 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.439 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.458 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.565 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.582 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.596 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.612 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.635 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.729 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.745 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.759 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.775 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.795 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.888 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.905 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.919 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.932 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.337.951 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.040 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.057 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.071 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.086 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.105 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.198 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.214 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.228 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.244 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.263 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.355 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.371 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.396 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.412 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.431 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.521 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.537 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.551 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.564 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.585 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.682 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.699 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.714 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.733 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.754 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.846 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.863 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.877 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.889 [runtime.cc:1673] 619759 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.907 [elf.cc:566] 619759 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.338.998 [runtime.cc:1688] 619759 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.015 [runtime.cc:1953] 619759 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.030 [runtime.cc:1897] 619759 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619423,python3.9):2025-04-03-09:35:50.339.059 [devmm_svm.c:1972][ascend][curpid:619423,619759][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.339.075 [hdc.cc:83] [619759][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.090 [api_impl.cc:1990] 619759 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.114 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.606 [api_impl.cc:1990] 619759 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.636 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.655 [api_impl.cc:2664] 619759 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619423,python3.9):2025-04-03-09:35:50.339.816 [devmm_svm.c:1972][ascend][curpid:619423,619759][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.339.835 [hdc.cc:83] [619759][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.850 [api_impl.cc:1990] 619759 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.865 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.885 [api_impl.cc:2096] 619759 HostMalloc: size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.339.906 [api_error.cc:1260] 619759 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.148 [api_impl.cc:1990] 619759 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.238 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.262 [api_impl.cc:2664] 619759 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.402 [api_impl.cc:1990] 619759 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.421 [api_error.cc:1218] 619759 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.440 [api_impl.cc:2664] 619759 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.467 [peterson_lock.cc:69] [619759][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.484 [peterson_lock.cc:47] [619759][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.518 [hccl_comm.cc:129] [619759]hcclCommInitInfo:commId[hccl-619423-0-1743644148-hccl_world_group], rank[1], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[1], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.539 [hccl_comm.cc:175] [619759][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.556 [topo_matcher.cc:528] [619759][SetAivMode]AivMode is set to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.569 [topo_matcher.cc:540] [619759][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.587 [hcom_plugin.cc:349] [619759][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.340.617 [runtime.cc:5239] 619759 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.633 [adapter_rts.cc:198] [619759][hrtGetDeviceRefresh]deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.646 [hcom_common.cc:85] [619759][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.670 [op_base.cc:128] [619759][HcclGetDeviceId] deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.687 [hccl_comm.cc:1296] [619759]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffebf7eccb8] topoSize[2] [INFO] GE(619423,python3.9):2025-04-03-09:35:50.340.724 [hcom_topo_info.cc:24]619759 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.738 [hcom_plugin.cc:371] [619759][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.766 [topoinfo_detect.cc:555] [619759]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.781 [topoinfo_detect.cc:321] [619759]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.795 [topoinfo_detect.cc:219] [619759]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.817 [op_base.cc:1050] [619759][InitCommRootInfo]Init success, rankNum[8], rank[1], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.834 [profiler_base.cc:342] [619759]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.851 [op_base.cc:1107] [619759][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4979700]us, rankNum[8], rank[1] [WARNING] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.340.916 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.340.942 [op_base.cc:853] [619759]HcclGetCommName input handle=0xfffeac022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.340.972 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 1 [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.340.995 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 1, group name: hccl_world_group [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.341.092 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.341.114 [device.cpp:183]619759 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 1. [INFO] DEVICE(619423,fffebe7cefa0,python3.9):2025-04-03-09:35:50.341.132 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 1, group name: hccl_world_group [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.341.147 [api_impl.cc:3419] 619759 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.341.161 [runtime.cc:3444] 619759 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.341.175 [api_impl.cc:3449] 619759 DeviceReset: Succ, userDeviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.341.186 [device.cpp:189]619759 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 1. [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.341.208 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:50.341.306 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5118.57 msec. [WARNING] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:50.341.363 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.341.431 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5620.34 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.341.465 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5620.39 msec. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.341.481 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.341.502 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9757.87 msec. [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.341.531 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.254 [device.cpp:143]619423 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.292 [api_impl.cc:2999] 619423 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.307 [device.cc:306] 619423 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.322 [runtime.cc:3393] 619423 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.337 [runtime.cc:3560] 619423 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.353 [api_impl.cc:2987] 619423 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.372 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.393 [api_impl.cc:2992] 619423 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.412 [api_impl.cc:3025] 619423 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.427 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.443 [acl.cpp:121]619423 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.461 [device.cpp:154]619423 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.356.499 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.520 [context.cpp:174]619423 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.356.542 [api_impl.cc:6913] 619423 CtxSetSysParamOpt: curCtx = 0xaaaaf7bf5f00, configOpt=0, configVal=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.356.572 [context.cpp:158]619423 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.357.116 [op_base.cc:1214] [619423][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.357.141 [op_base.cc:128] [619423][HcclGetDeviceId] deviceLogicId[1] [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.357.166 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.357.185 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.357.203 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.357.245 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.357.730 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.148 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.222 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.390 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 177.761 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.433 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.474 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 38.94 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.499 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.519 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 20.79 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.537 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.602 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 61.93 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.632 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.489 msec. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.728 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.787 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.31 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.864 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 47.37 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.905 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.174 msec. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.947 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.358.982 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.026 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 42.12 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.049 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.080 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 29.28 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.097 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 17.49 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.131 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.161 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 27.56 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.207 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.256 msec. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.238 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.352 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.431 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 76.98 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.490 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 34.21 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.535 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 19.38 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.584 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 25.41 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.646 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 37.291 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.781 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 108.1 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.809 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 2.2 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.839 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.911 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 53.71 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.963 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 25.31 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.359.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 43.62 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.630 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 550.691 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.659 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.718 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 34.46 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.765 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.524 msec. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.785 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.360.833 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 2.104 msec. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.209 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 47.95 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.236 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.309 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 69.66 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.332 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.385 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 50.08 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.406 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.438 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 27.01 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.483 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.506 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 44.17 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.548 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 5.3 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.604 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.626 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 54.47 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.645 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.662 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 17.33 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.679 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.696 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 15.561 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.731 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.749 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 35.7 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.769 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.785 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 15.64 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.801 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.817 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 14.7 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.832 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.873 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 37.49 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.922 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.940 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 47.77 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.957 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.975 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 17.07 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.361.999 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.014 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 13.86 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.029 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.046 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 16.18 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.064 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.080 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 15.07 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.115 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.133 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 35 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.147 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.207 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 56.51 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.246 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.265 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 35.06 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.286 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.303 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 16.83 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.319 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.336 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 15.34 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.351 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.368 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 15.25 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.383 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.400 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 17.44 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.432 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 38.74 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.487 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.504 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 29.27 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.522 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.539 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 16.58 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.556 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.575 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 18.09 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.654 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 60.46 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.911 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 227.94 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.935 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.362.988 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 29.59 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.009 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.076 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 46.671 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.098 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.140 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 23.24 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.162 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.211 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 27.93 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.228 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.267 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 22.81 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.295 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.375 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 60.32 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.397 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.464 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 46.72 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.483 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 89.09 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.614 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 52.18 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.707 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.726 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.843 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 115.971 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.865 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.363.884 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.009 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 123.31 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.033 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 64.1 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.243 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 58.21 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.281 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.299 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.313 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.343 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.357 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.374 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.389 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.403 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.418 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.446 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.461 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.478 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.492 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.729 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 218.5 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.754 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.771 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.797 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.815 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.830 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.929 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 77.66 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.952 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.968 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.364.985 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.365.060 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 55.68 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.365.118 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.232 msec. [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.365.459 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.365.603 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.365.905 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 237.441 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.033 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 33.96 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.142 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 77.31 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.185 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 15.37 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.238 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 27.07 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.357 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 94.03 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.407 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 25.35 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.454 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 21.47 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.552 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 59.41 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.581 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 6.39 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.624 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.651 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.677 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.074 msec. [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.366.718 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.047 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.086 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.38 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.103 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.119 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.241 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 53.22 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 59.91 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.396 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 43.25 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.488 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 67.03 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.511 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.529 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.585 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 37.62 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.623 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 14.47 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.648 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 0.5 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.688 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.546 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.747 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.607 msec. [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.785 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.895 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.920 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 0.97 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.967 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 28.07 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.367.994 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 4.92 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.016 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.47 us [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.044 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 84.36 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 1.39 us [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.187 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.32 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.246 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.475 msec. [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.279 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.366 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.096 msec. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.368.820 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.368.852 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.368.868 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.368.907 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.149 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.209 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.302 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.410 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.12 msec. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.513 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.565 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.588 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.607 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.622 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.641 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.369.684 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.369.958 [context.cpp:72]619793 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.369.993 [api_impl.cc:3717] 619793 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.370.016 [awatchdog_core.c:175](tid:619793) create watchdog for id : 65543, tid : 619793, timeout : 300s [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.370.021 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.370.035 [context.cpp:80]619793 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.370.079 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.370.125 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.370.149 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.371.600 [api_impl.cc:7115] 619793 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5a00000 [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.379.812 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.379.852 [api_impl.cc:7129] 619793 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.379.965 [api_impl.cc:7145] 619793 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.054 [device.cpp:143]619793 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.076 [api_impl.cc:2999] 619793 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.091 [device.cc:306] 619793 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.104 [runtime.cc:3393] 619793 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.119 [runtime.cc:3560] 619793 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.132 [api_impl.cc:2987] 619793 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.143 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.168 [api_impl.cc:2992] 619793 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.187 [api_impl.cc:3025] 619793 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.203 [acl.cpp:115]619793 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.216 [acl.cpp:121]619793 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.230 [device.cpp:154]619793 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.380.259 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.280 [context.cpp:174]619793 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.298 [api_impl.cc:6913] 619793 CtxSetSysParamOpt: curCtx = 0xaaaaf7bf5f00, configOpt=0, configVal=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.311 [context.cpp:158]619793 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.380.380 [op_base.cc:1214] [619793][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.401 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.380.415 [adapter_rts.cc:234] [619793][hrtGetDevice]deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:50.380.426 [op_base.cc:128] [619793][HcclGetDeviceId] deviceLogicId[1] [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.458 [context.cpp:72]619793 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.475 [api_impl.cc:3717] 619793 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.494 [context.cpp:80]619793 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.520 [api_error.cc:1807] 619793 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.541 [memory_task.cc:342] 619793 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.558 [memory_task.cc:473] 619793 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.572 [memory_task.cc:518] 619793 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.607 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.626 [memory_task.cc:1007] 619793 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaab0371cf30, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.643 [memory_task.cc:1166] 619793 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.657 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.674 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.698 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.716 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.740 [callback.cpp:97]619793 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.772 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.791 [task_info.cc:521] 619793 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.807 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.821 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.835 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.848 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.861 [callback.cpp:109]619793 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.882 [npu_driver.cc:3844] 619598 CqReportGet: cq report get start, device_id=1, tsId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.905 [npu_driver.cc:3851] 619598 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.917 [api_impl.cc:5087] 619598 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.380.933 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.937 [callback.cpp:138]619598 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.380.960 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.380.971 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.380.976 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] DEVICE(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.380.990 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61029220352, align_size:61029220352 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.382.310 [api_impl.cc:7115] 619793 ReserveMemAddress: device malloc Succ, size=61029220352(bytes) start ptr=0x12d100000000, end ptr=0x12df35a00000 [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:50.386.773 [awatchdog_monitor.c:160](tid:619461) thread 619759 not exist [INFO] PRE_ACT(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.392.013 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.058 [api_impl.cc:7129] 619793 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.175 [api_impl.cc:7145] 619793 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.392.373 [device.cpp:231]619793 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.399 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.392.419 [acl.cpp:802]619793 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.528 [npu_driver.cc:404] 619793 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.570 [stream.cc:2246] 619793 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.608 [stream_sqcq_manage.cc:147] 619793 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.392.624 [npu_driver.cc:882] 619793 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.125 [npu_driver.cc:6125] 619793 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.181 [npu_driver.cc:6137] 619793 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=414, streamId=6, logicCq=414, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.206 [npu_driver.cc:6189] 619793 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=6, logicCq=414. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.227 [npu_driver.cc:5916] 619793 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc55f000, addr=dfffcc55f000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.240 [stream.cc:653] 619793 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.280 [stream.cc:276] 619793 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.300 [stream.cc:688] 619793 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=1, streamResId=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.393.317 [acl_resource_manager.cpp:814]619793 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.393.330 [acl_resource_manager.cpp:816]619793 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.472 [engine.cc:188] 619793 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.492 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.515 [task_info.cc:409] 619793 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.538 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.552 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.569 [npu_driver.cc:751] 619793 SqTaskSend: device_id=1, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.584 [stars_engine.cc:893] 619793 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.600 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.612 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.625 [stream.cc:2886] 619793 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.637 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.647 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.658 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.672 [logger.cc:152] 619793 TaskFinished: device_id=1, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.688 [stars_engine.cc:437] 619793 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.706 [stream.cc:2886] 619793 StarsWaitForTask: No.1 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.721 [stream.cc:852] 619793 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.735 [stream.cc:938] 619793 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.747 [stream.cc:1507] 619793 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.772 [stream_sqcq_manage.cc:451] 619793 FreeLogicCq: Return(0), threadIdentifier(281469599543200), devId(1), tsId(0), cqId(414), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.794 [npu_driver.cc:6163] 619793 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=6, logicCq=414. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.823 [npu_driver.cc:1180] 619793 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=414, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.840 [stream.cc:237] 619793 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.856 [stream_sqcq_manage.cc:216] 619793 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.393.871 [npu_driver.cc:966] 619793 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.394.384 [npu_driver.cc:974] 619793 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.394.403 [stream_sqcq_manage.cc:228] 619793 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.394.418 [npu_driver.cc:432] 619793 StreamIdFree: stream id free begin, device_id=1, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.394.453 [npu_driver.cc:441] 619793 StreamIdFree: Stream id free success, stream_id=6, device_id=1, tsId=0, streamFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.394.469 [device_msg_handler.cc:110] 619793 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.395.207 [device_msg_handler.cc:127] 619793 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.395.260 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.395.296 [npu_driver.cc:500] 619793 GetFaultEvent: get fault event, device_id=1, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.395.373 [acl.cpp:782]619793 GetFaultEventInfo: can not get fault event of device 1, runtime errorCode is 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:50.395.400 [error_manager.cc:989]619793 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.395.709 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.395.749 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.395.768 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.395.783 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.396.081 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.215 [context.cpp:72]619793 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.243 [api_impl.cc:3717] 619793 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.261 [context.cpp:80]619793 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.281 [api_error.cc:1807] 619793 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.299 [memory_task.cc:342] 619793 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.317 [memory_task.cc:473] 619793 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.330 [memory_task.cc:518] 619793 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.342 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.396.344 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.360 [memory_task.cc:1007] 619793 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaab0371d950, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.375 [memory_task.cc:1166] 619793 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.407 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.422 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.440 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.455 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.473 [callback.cpp:97]619793 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.486 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.502 [task_info.cc:521] 619793 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.518 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.530 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.546 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.558 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.568 [callback.cpp:109]619793 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.586 [npu_driver.cc:3844] 619598 CqReportGet: cq report get start, device_id=1, tsId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.608 [npu_driver.cc:3851] 619598 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.621 [api_impl.cc:5087] 619598 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.640 [callback.cpp:138]619598 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.641 [device.cpp:231]619793 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.654 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.661 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.668 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.396.673 [acl.cpp:802]619793 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.705 [npu_driver.cc:404] 619793 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.731 [stream.cc:2246] 619793 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.765 [stream_sqcq_manage.cc:147] 619793 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.396.795 [npu_driver.cc:882] 619793 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.046 [npu_driver.cc:6125] 619793 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.087 [npu_driver.cc:6137] 619793 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=415, streamId=7, logicCq=415, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.108 [npu_driver.cc:6189] 619793 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=7, logicCq=415. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.123 [npu_driver.cc:5916] 619793 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc55f000, addr=dfffcc55f000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.138 [stream.cc:653] 619793 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.175 [stream.cc:276] 619793 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.191 [stream.cc:688] 619793 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=1, streamResId=65534 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.397.206 [acl_resource_manager.cpp:814]619793 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.397.221 [acl_resource_manager.cpp:816]619793 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.265 [engine.cc:188] 619793 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.284 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.299 [task_info.cc:409] 619793 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.309 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.322 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.340 [npu_driver.cc:751] 619793 SqTaskSend: device_id=1, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.352 [stars_engine.cc:893] 619793 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.368 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.384 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.400 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.413 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.427 [logger.cc:152] 619793 TaskFinished: device_id=1, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.446 [stars_engine.cc:437] 619793 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.463 [stream.cc:2886] 619793 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.478 [stream.cc:852] 619793 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.508 [stream.cc:938] 619793 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.522 [stream.cc:1507] 619793 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.543 [stream_sqcq_manage.cc:451] 619793 FreeLogicCq: Return(0), threadIdentifier(281469599543200), devId(1), tsId(0), cqId(415), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.564 [npu_driver.cc:6163] 619793 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=7, logicCq=415. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.592 [npu_driver.cc:1180] 619793 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=415, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.609 [stream.cc:237] 619793 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.624 [stream_sqcq_manage.cc:216] 619793 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.397.638 [npu_driver.cc:966] 619793 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.097 [npu_driver.cc:974] 619793 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.114 [stream_sqcq_manage.cc:228] 619793 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.130 [npu_driver.cc:432] 619793 StreamIdFree: stream id free begin, device_id=1, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.147 [npu_driver.cc:441] 619793 StreamIdFree: Stream id free success, stream_id=7, device_id=1, tsId=0, streamFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.163 [device_msg_handler.cc:110] 619793 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.452 [device_msg_handler.cc:127] 619793 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.489 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.398.513 [npu_driver.cc:500] 619793 GetFaultEvent: get fault event, device_id=1, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.398.575 [acl.cpp:782]619793 GetFaultEventInfo: can not get fault event of device 1, runtime errorCode is 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:50.398.594 [error_manager.cc:989]619793 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.398.890 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.398.924 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.398.944 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.398.961 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.399.034 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.120 [context.cpp:72]619793 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.155 [api_impl.cc:3717] 619793 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.172 [context.cpp:80]619793 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.194 [api_error.cc:1807] 619793 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.215 [memory_task.cc:342] 619793 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.229 [memory_task.cc:473] 619793 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.241 [memory_task.cc:518] 619793 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.256 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.271 [memory_task.cc:1007] 619793 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaab0371e3c0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.287 [memory_task.cc:1166] 619793 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.299 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.313 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.330 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.345 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.363 [callback.cpp:97]619793 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.381 [task_res.cc:380] 619793 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.398 [task_info.cc:521] 619793 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.415 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.428 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.443 [task_submit.cc:573] 619793 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.456 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.467 [callback.cpp:109]619793 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.482 [npu_driver.cc:3844] 619598 CqReportGet: cq report get start, device_id=1, tsId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.497 [npu_driver.cc:3851] 619598 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.525 [api_impl.cc:5087] 619598 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.533 [device.cpp:231]619793 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.539 [callback.cpp:138]619598 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.549 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.554 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.399.562 [acl.cpp:802]619793 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.570 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.591 [npu_driver.cc:404] 619793 StreamIdAlloc: stream id alloc begin, device_id=1, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.621 [stream.cc:2246] 619793 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.658 [stream_sqcq_manage.cc:147] 619793 Alloc: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.674 [npu_driver.cc:882] 619793 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.919 [npu_driver.cc:6125] 619793 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.964 [npu_driver.cc:6137] 619793 LogicCqAllocateV2: success: deviceId=1, tsId=0, cqId=416, streamId=8, logicCq=416, cqeDepth=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.399.986 [npu_driver.cc:6189] 619793 StreamBindLogicCq: success: deviceId=1, tsId=0, streamId=8, logicCq=416. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.002 [npu_driver.cc:5916] 619793 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=cc55f000, addr=dfffcc55f000size=4096 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.018 [stream.cc:653] 619793 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.058 [stream.cc:276] 619793 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.074 [stream.cc:688] 619793 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=1, streamResId=65534 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.400.092 [acl_resource_manager.cpp:814]619793 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.400.107 [acl_resource_manager.cpp:816]619793 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.149 [engine.cc:188] 619793 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.181 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.200 [task_info.cc:409] 619793 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.215 [stream.cc:1400] 619793 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.229 [stream.cc:2595] 619793 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.245 [npu_driver.cc:751] 619793 SqTaskSend: device_id=1, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.274 [stars_engine.cc:893] 619793 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.289 [stream.cc:1494] 619793 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.304 [npu_driver.cc:5547] 619793 GetSqHead: deviceId=1, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.319 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.336 [task.cc:185] 619793 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.352 [logger.cc:152] 619793 TaskFinished: device_id=1, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.372 [stars_engine.cc:437] 619793 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.388 [stream.cc:2886] 619793 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.401 [stream.cc:852] 619793 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.417 [stream.cc:938] 619793 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.430 [stream.cc:1507] 619793 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.452 [stream_sqcq_manage.cc:451] 619793 FreeLogicCq: Return(0), threadIdentifier(281469599543200), devId(1), tsId(0), cqId(416), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.473 [npu_driver.cc:6163] 619793 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=8, logicCq=416. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.502 [npu_driver.cc:1180] 619793 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=416, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.518 [stream.cc:237] 619793 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.531 [stream_sqcq_manage.cc:216] 619793 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.545 [npu_driver.cc:966] 619793 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.400.994 [npu_driver.cc:974] 619793 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.013 [stream_sqcq_manage.cc:228] 619793 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.032 [npu_driver.cc:432] 619793 StreamIdFree: stream id free begin, device_id=1, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.049 [npu_driver.cc:441] 619793 StreamIdFree: Stream id free success, stream_id=8, device_id=1, tsId=0, streamFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.067 [device_msg_handler.cc:110] 619793 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.362 [device_msg_handler.cc:127] 619793 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.394 [runtime.cc:5239] 619793 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.401.416 [npu_driver.cc:500] 619793 GetFaultEvent: get fault event, device_id=1, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.401.483 [acl.cpp:782]619793 GetFaultEventInfo: can not get fault event of device 1, runtime errorCode is 0 [INFO] GE(619423,python3.9):2025-04-03-09:35:50.401.501 [error_manager.cc:989]619793 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.401.796 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.401.829 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.401.849 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.401.866 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619423,fffebf7eefa0,python3.9):2025-04-03-09:35:50.401.937 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.500.175 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.500.204 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.500.223 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.500.243 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.500.261 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.500.277 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.500.294 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.176 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.180 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.182 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.186 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.247 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.532.251 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.326 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.347 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.357 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.391 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.419 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.440 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.441 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.440 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.465 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.466 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.467 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.451 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.477 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.482 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.494 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.495 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.496 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.497 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.508 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.513 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.520 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.525 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.547 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.551 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.570 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.544 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.571 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.588 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.614 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.635 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.653 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.532.660 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.674 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.677 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.690 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.532.696 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.916.189 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:50.916.226 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.916.247 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:50.916.267 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.916.287 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.916.306 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.916.323 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.977.717 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.759 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.977.778 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.977.820 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.855 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.876 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.940 [stars_engine.cc:643] 619423 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.972 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.977.990 [task_manager.cc:485] 619423 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.007 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.035 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.050 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.070 [stars_engine.cc:643] 619423 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.086 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.102 [task_manager.cc:485] 619423 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.114 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.129 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.141 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.159 [stars_engine.cc:643] 619423 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.172 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.185 [task_manager.cc:485] 619423 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.198 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.215 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.228 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.251 [stars_engine.cc:622] 619423 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=410, retCode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.267 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.283 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.978.297 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.978.314 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.329 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.344 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.358 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.379 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.394 [stream.cc:2886] 619423 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.407 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.421 [stream.cc:2886] 619423 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.432 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.445 [stream.cc:2886] 619423 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.456 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.470 [stream.cc:2886] 619423 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.486 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.497 [stream.cc:2886] 619423 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.509 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.522 [stream.cc:2886] 619423 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.534 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.545 [stream.cc:2886] 619423 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.558 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.573 [stream.cc:2886] 619423 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.586 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.597 [stream.cc:2886] 619423 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.609 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.620 [stream.cc:2886] 619423 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.634 [event.cc:310] 619423 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.651 [task_res.cc:380] 619423 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=1, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.675 [event.cc:249] 619423 UpdateLatestRecord: device_id=1, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.689 [event_task.cc:259] 619423 ConstructSqeForEventRecordTask: event_record: device_id=1, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.701 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.713 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.729 [task_submit.cc:573] 619423 AllocTaskAndSendStars: device_id=1, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.751 [stars_engine.cc:1122] 619423 SyncTask: Task Wait: device_id=1, stream_id=3, task_id=0, logicCqId=411. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.768 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.781 [stars_engine.cc:1071] 619423 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.795 [stars_engine.cc:1017] 619423 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.812 [stars_engine.cc:1027] 619423 ProcReport: taskType=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.824 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.838 [event.cc:249] 619423 UpdateLatestRecord: device_id=1, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.853 [event_task.cc:211] 619423 DoCompleteSuccessForEventRecordTask: event_record: device_id=1, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea571459b1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.867 [event.cc:122] 619423 TryFreeEventIdAndCheckCanBeDelete: device_id=1, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.881 [task_res.cc:323] 619423 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=1, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.901 [npu_driver.cc:5618] 619423 GetSqEnable: deviceId=1, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.914 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.978.925 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.978.938 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.978.981 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.000 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.014 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.029 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.043 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.056 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.070 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.082 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.092 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.104 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.133 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.148 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.171 [stars_engine.cc:622] 619423 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=411, retCode=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.186 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:50.979.198 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.210 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:50.979.225 [memory.cpp:421]619423 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.72s ============================================================================================================== 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340822] [debug] [619759] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340829] [debug] [619759] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340835] [debug] [619759] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340842] [debug] [619759] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360570] [debug] [619759] [lcal_comm.cpp:254] InitCommMem 1/8, localRank_ : 1, localRankSize_ : 8 success [2025-04-03 09:35:45.360713] [info] [619759] [lcal_comm.cpp:259] LcalCommInit 1/8 success and extraFlag:0 commArgs_.localRank : 1 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.392343] [error] [619793] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.396618] [error] [619793] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.399513] [error] [619793] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.169 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.217 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:50.988.291 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619423:281473725853728,MainProcess):2025-04-03-09:35:50.988.451 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.524 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.599 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.640 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:50.988.667 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.012.182 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.012.209 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.012.228 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.012.242 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.012.263 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.012.282 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.012.303 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.024 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.114 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619423,ffff54dc1fa0,python3.9):2025-04-03-09:35:51.088.214 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.324 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.347 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619423,ffff555d1fa0,python3.9):2025-04-03-09:35:51.088.406 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.488 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.520 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.578 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619423,ffff4f7eefa0,python3.9):2025-04-03-09:35:51.088.658 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.793 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.088.814 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619423,ffff4fffefa0,python3.9):2025-04-03-09:35:51.088.854 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.080 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.099 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.123 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.143 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.269 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.294 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.312 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.342 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.362 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.382 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.400 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.416 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.431 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.447 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.466 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.482 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.497 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.513 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.527 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.547 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.576 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.589 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.604 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.619 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.634 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.647 [stream.cpp:131]619423 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.664 [api_impl.cc:1394] 619423 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.676 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.691 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.089.706 [api_error.cc:895] 619423 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.089.717 [stream.cpp:146]619423 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.764 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.824 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.879 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.898 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.089.915 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.087 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.108 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.822 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.847 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.866 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.884 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.090.899 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.113 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.136 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.164 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.214 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.233 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.091.253 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_1 [INFO] GE_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.101.489 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.101.524 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.101.549 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.565 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.587 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.604 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.616 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.628 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.639 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.660 [device.cpp:143]619423 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.680 [api_impl.cc:2999] 619423 SetDevice: device_id=1, deviceMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.692 [device.cc:306] 619423 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.705 [runtime.cc:3393] 619423 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.724 [runtime.cc:3560] 619423 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.742 [api_impl.cc:2987] 619423 NewDevice: SetCurRef = 0xaaaaf7bf5f00,userDeviceId=1,deviceId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.778 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.799 [api_impl.cc:2992] 619423 NewDevice: New device success, user_device_id=1, die_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.101.816 [api_impl.cc:3025] 619423 SetDevice: SetDevice success, curCtx=0xaaaaf7bf5f00, user_device_id=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.832 [acl.cpp:115]619423 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.845 [acl.cpp:121]619423 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.101.861 [device.cpp:154]619423 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:51.101.887 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:51.156.098 [awatchdog_monitor.c:160](tid:619461) thread 619793 not exist [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.360.182 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.360.301 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.360.322 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.360.337 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.360.352 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.360.363 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.360.372 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.364.180 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.364.182 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.364.183 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.275 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.293 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.307 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.310 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.364.305 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.318 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.364.317 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.327 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.335 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.347 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.369 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.376 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.379 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.385 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.391 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.421 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.424 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.436 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.436 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.441 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.460 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.468 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.467 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.480 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.503 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.506 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.518 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.364.524 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.535 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.537 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.553 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.364.556 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.428.187 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.428.219 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.428.239 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.428.256 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.428.273 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.428.290 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.428.314 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.524.178 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.524.202 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.524.223 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.524.237 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.524.250 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.524.267 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.524.281 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.532.737 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:51.532.765 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:51.532.784 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:51.532.799 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.532.822 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.532.917 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.940.168 [trs_cb_event.c:119][ascend][curpid:619423,619598][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =320; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:51.940.199 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.940.221 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:51.940.236 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.940.251 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:51.940.267 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:51.940.284 [npu_driver.cc:3828] 619598 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=320. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.036.202 [trs_cb_event.c:119][ascend][curpid:619423,619599][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=1, tid =321; ret=16) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.036.222 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.036.238 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.036.268 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.036.280 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.036.293 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.036.307 [npu_driver.cc:3828] 619599 CqReportIrqWait: Irq wait start, deviceId=1, tsId=0, groupId=321. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.177 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.189 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.187 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.187 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.188 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.196.194 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.293 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.313 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.318 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.327 [tensor_data_transfer.cpp:1054]619655 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.335 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.344 [tensor_data_transfer.cpp:1056]619655 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.350 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.354 [tensor_data_transfer.cpp:1054]619661 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.365 [api_impl.cc:6121] 619655 MemQueuePeek: Start to peek queue on devId 1, qid is 0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.373 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.381 [tensor_data_transfer.cpp:1056]619661 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.381 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.392 [npu_driver.cc:4330] 619655 MemQueuePeek: peek mem queue, deviceId=1, qid=0, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.403 [tensor_data_transfer.cpp:1054]619657 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.408 [api_impl.cc:6121] 619661 MemQueuePeek: Start to peek queue on devId 1, qid is 3. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.415 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.415 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.444 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.456 [tensor_data_transfer.cpp:1056]619657 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.463 [npu_driver.cc:4330] 619661 MemQueuePeek: peek mem queue, deviceId=1, qid=3, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.470 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.476 [tensor_data_transfer.cpp:1054]619659 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_tensor_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.488 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.496 [api_impl.cc:6121] 619657 MemQueuePeek: Start to peek queue on devId 1, qid is 1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.508 [tensor_data_transfer.cpp:1054]619665 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_histogram_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.512 [tensor_data_transfer.cpp:1056]619659 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.524 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.532 [npu_driver.cc:4330] 619657 MemQueuePeek: peek mem queue, deviceId=1, qid=1, timeout=800. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.540 [tensor_data_transfer.cpp:1056]619665 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.547 [api_impl.cc:6121] 619659 MemQueuePeek: Start to peek queue on devId 1, qid is 2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.196.556 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.571 [api_impl.cc:6121] 619665 MemQueuePeek: Start to peek queue on devId 1, qid is 5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.577 [npu_driver.cc:4330] 619659 MemQueuePeek: peek mem queue, deviceId=1, qid=2, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.588 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.597 [npu_driver.cc:4330] 619665 MemQueuePeek: peek mem queue, deviceId=1, qid=5, timeout=800. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.196.612 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.382 [op_base.cc:2136] [619423]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619423,fffebe7cefa0,python3.9):2025-04-03-09:35:52.341.406 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.341.419 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.448 [adapter_rts.cc:198] [619423][hrtGetDeviceRefresh]deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.483 [op_base.cc:2151] [619423][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.509 [op_base.cc:128] [619423][HcclGetDeviceId] deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.558 [op_base.cc:2168] [619423]com is not global com [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.578 [hcom_plugin.cc:381] [619423][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.628 [hccl_comm.cc:1375] [619423]UnRegistTaskAbortHandler begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.647 [task_abort_handler.cc:140] [619423]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.341.664 [api_impl.cc:4192] 619423 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.341.682 [runtime.cc:4698] 619423 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.699 [hccl_communicator.cc:2112] [619423][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.341.712 [hccl_communicator.cc:2114] [619423][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.728 [hccl_communicator.cc:2082] [619423]start to stop the backGround Thread [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.341.742 [heartbeat.cc:1203] [619423]Heartbeat has been destroyed [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.755 [heartbeat.cc:451] [619423][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.341.767 [heartbeat.cc:452] [619423]Heartbeat has been destroyed [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.806 [hccl_impl.cc:69] [619423]start hccl resources destruction:deviceLogicId[1] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.341.842 [heartbeat.cc:1203] [619423]Heartbeat has been destroyed [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.341.856 [heartbeat.cc:451] [619423][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.341.869 [heartbeat.cc:452] [619423]Heartbeat has been destroyed [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.342.360 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.342.809 [task_exception_handler.cc:1232] [619423]deInit taskFailCallback [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.343.638 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.343.662 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.343.753 [api_error.cc:1278] 619423 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.343.772 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.343.806 [hccl_socket_manager.cc:81] [619423][DeInit][Server]ip[192.168.2.198] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.343.825 [hccl_socket_manager.cc:83] [619423][DeInit][Server]ip[192.168.2.198] port[16666] [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.343.842 [hccl_socket.cc:232] [619423][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.343.857 [network_manager.cc:69] [619423]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.343.871 [network_manager.cc:458] [619423]NetworkManager devicePhyId_[1] vnic stopped. refcount[0] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:52.343.918 [ra_host.c:911]tid:619423,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[1], local_ip[192.168.2.198] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:52.344.304 [ra_host.c:404]tid:619423,ra_socket_deinit(404) : Input parameters: phy_id[1] family[2] local_ip[192.168.2.198] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.344.465 [hccl_socket.cc:68] [619423][HcclSocket][DeInit] device[1] stops listen on ip[192.168.2.198], port[16666] success, socketType[0]. [WARNING] HCCL(619423,python3.9):2025-04-03-09:35:52.344.488 [hccl_socket.cc:232] [619423][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.344.512 [hccl_communicator.cc:1741] [619423][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.344.530 [network_manager.cc:69] [619423]NetworkManager::GetInstance deviceLogicID[1]. [INFO] HCCP(619423,python3.9):2025-04-03-09:35:52.344.545 [ra_host.c:248]tid:619423,ra_deinit(248) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(619423,python3.9):2025-04-03-09:35:52.344.562 [ra_hdc.c:651]tid:619423,ra_hdc_deinit(651) : hdc deinit start! phy_id[1] [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.344.659 [hdc_client.c:417][ascend][curpid:619423,619423][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4277) [INFO] ROCE(619423,python3.9):2025-04-03-09:35:52.344.730 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619423,python3.9):2025-04-03-09:35:52.344.746 [ra_hdc.c:685]tid:619423,ra_hdc_deinit(685) : hdc deinit OK! phy_id[1] [INFO] ROCE(619423,python3.9):2025-04-03-09:35:52.344.777 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.829 [tsd_client.cpp:189][TsdCapabilityGet][tid:619423] TsdCapabilityGet Begin. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.845 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.857 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.874 [process_mode_manager.cpp:1042][CapabilityGet][tid:619423] [ProcessModeManager] enter into CapabilityGet process deviceId[1] type[6] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.344.891 [adapter_tdt.cc:66] [619423]Get TsdCapability success. deviceLogicId[1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.909 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.925 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.937 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619423] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.344.958 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619423] add close subproc:31579, proctype:0 [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.345.001 [hdc_client.cpp:154][TsdRecvData][tid:619423] recv timeout is:150000 ms [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.417.963 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619423] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.417.987 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619423] [TsdClient] DeviceMsgProc recvMsg realDeviceId[1] msgType[61] localDevId[1] rspCode[0]heterogeneousSubPid[31579], tsdSupportLevel_[63] [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.418.005 [hdc_client.cpp:172][TsdRecvData][tid:619423] [deviceId=1] the Tsd recv data already [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.418.021 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619423] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.418.050 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619423] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.065 [adapter_tdt.cc:52] [619423]Open TsdProcess success. deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.078 [network_manager.cc:405] [619423][DeInit]finish CloseHccpProcess, phy_id[1], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.095 [network_manager.cc:409] [619423]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.111 [hccl_communicator_attrs.cc:1307] [619423][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.128 [hccl_communicator_attrs.cc:1275] [619423]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.146 [notify_pool_impl.cc:43] [619423]NotifyPoolImpl Destroy. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.169 [notify_pool_impl.cc:56] [619423]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.194 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.209 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.221 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.237 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.251 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.266 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.279 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.295 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.320 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.336 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.356 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=0. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.418.381 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=0.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.400 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.413 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.426 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.441 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.454 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.467 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.481 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.493 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.515 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.530 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.545 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.558 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=2. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.418.575 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=2.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.591 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.603 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.615 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.629 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.641 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.654 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.670 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.685 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=3. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.418.700 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=3.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.716 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.728 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.741 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.754 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.768 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.782 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.794 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.807 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=4. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.418.822 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=4.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.838 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.850 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.862 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.874 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.890 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.905 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.418.930 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.944 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=5. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.418.959 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=5.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.973 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.984 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.418.993 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.002 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.012 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.028 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.042 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.057 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=6. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.419.074 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=6.) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.088 [api_impl.cc:3100] 619423 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.102 [api_impl.cc:3111] 619423 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.113 [api_impl.cc:3130] 619423 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.124 [p2p_mgmt.cc:171] [619423]local logic id:1, local physic id:1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.135 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.148 [api_impl.cc:3520] 619423 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.165 [p2p_mgmt.cc:178] [619423]disable p2p: local logic id:1, remote physic id:7. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.178 [api_impl.cc:3184] 619423 DisableP2P: Disable P2P devIdDes=1, phyIdSrc=7. [INFO] DRV(619423,python3.9):2025-04-03-09:35:52.419.192 [devdrv_pcie.c:890][ascend][curpid:619423,619423][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=1; peer_phy_devid=7.) [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.207 [hccl_communicator.cc:2043] [619423]Profiling is deinitiated. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.219 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.239 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.419.253 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.270 [opexecounter.cc:103] [619423]free counter mem resource [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.419.284 [atrace_api.c:118](tid:619423) AtraceDestroy start. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.419.303 [tracer_mgr_operate.c:248](tid:619423) destroy object HCCL_619759_1, exitSave(true). [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.419.320 [atrace_api.c:120](tid:619423) AtraceDestroy end. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.419.332 [adapter_trace.cc:33] [619423]Call TraceDestroy, Params:handle[0xaaaae6f988f8] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.422.639 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.422.681 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.422.736 [queue_notify_manager.cc:100] [619423]QueueNotifyManager Destroy. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.422.754 [queue_notify_manager.cc:106] [619423]QueueNotifyManager Destroy success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.072 [queue_notify_manager.cc:100] [619423]QueueNotifyManager Destroy. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.090 [queue_notify_manager.cc:106] [619423]QueueNotifyManager Destroy success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.207 [ccl_buffer_manager.cc:125] [619423][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.226 [ccl_buffer_manager.cc:147] [619423][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.243 [ccl_buffer_manager.cc:160] [619423][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.260 [ccl_buffer_manager.cc:163] [619423][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.273 [ccl_buffer_manager.cc:166] [619423][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.329 [op_base.cc:128] [619423][HcclGetDeviceId] deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.364 [op_base.cc:2196] [619423]op_base comm destroy complete, take time [81947]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[1]. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.423.386 [device.cpp:183]619423 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.405 [api_impl.cc:3419] 619423 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.418 [runtime.cc:3444] 619423 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.433 [api_impl.cc:3449] 619423 DeviceReset: Succ, userDeviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.423.445 [device.cpp:189]619423 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 1. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.423.490 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.423.517 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.423.532 [hcom_executor.cc:40] [619423]Hcom Excutor Finalize start. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.623 [api_impl.cc:3419] 619667 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.636 [api_impl.cc:3419] 619669 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.643 [runtime.cc:3444] 619667 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.665 [runtime.cc:3444] 619669 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.668 [api_impl.cc:3449] 619667 DeviceReset: Succ, userDeviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.683 [api_impl.cc:3449] 619669 DeviceReset: Succ, userDeviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.945 [api_impl.cc:3419] 619668 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.958 [api_impl.cc:3419] 619670 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.984 [runtime.cc:3444] 619668 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.423.996 [runtime.cc:3444] 619670 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.003 [api_impl.cc:3449] 619668 DeviceReset: Succ, userDeviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.015 [api_impl.cc:3449] 619670 DeviceReset: Succ, userDeviceId=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.217 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.424.244 [adapter_rts.cc:321] [619423][hrtCtxSetCurrent] success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.424.259 [stream.cc:182] [619423]Switch Ctx ret[0], curCtx[0xaaaaf7bf5f00], setCtx[0xaaaaf7bf5f00], stream id[5] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.276 [api_impl.cc:1228] 619423 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.424.295 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.424.338 [single_op_manager.cc:50]619423 ReleaseResource:ReleaseResource in. resource id = 0xaaab02f9a020 [INFO] GE(619423,python3.9):2025-04-03-09:35:52.424.361 [caching_allocator.cc:402]619423 TryFreeBlocks:Try free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.424.377 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:52.424.414 [npu_memory_allocator.cc:72]619423 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.424.429 [stream_executor.cpp:197]619423 RemoveExecutor: To remove executor by stream = 187651466043424 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.424.470 [acl_resource_manager.cpp:823]619423 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.488 [stream.cc:852] 619423 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.505 [stream.cc:938] 619423 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.521 [stream.cc:1507] 619423 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.545 [stream_sqcq_manage.cc:451] 619423 FreeLogicCq: Return(0), threadIdentifier(281473725853728), devId(1), tsId(0), cqId(413), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.571 [npu_driver.cc:6163] 619423 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=5, logicCq=413. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.424.608 [npu_driver.cc:1180] 619423 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=413, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.428.821 [stream.cc:237] 619423 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.428.852 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.428.868 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.429.543 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.429.564 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.429.594 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.429.613 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=5, device_id=1, tsId=0, streamFlag=1536 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.430.292 [stream.cc:185] [619423][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab02f9a020], stream id[5] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.312 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.430.334 [adapter_rts.cc:321] [619423][hrtCtxSetCurrent] success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.430.347 [stream.cc:193] [619423]Restore Ctx ret[0], setCtx[0xaaaaf7bf5f00], stream id[5] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.401 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.430.420 [adapter_rts.cc:321] [619423][hrtCtxSetCurrent] success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.430.435 [stream.cc:182] [619423]Switch Ctx ret[0], curCtx[0xaaaaf7bf5f00], setCtx[0xaaaaf7bf5f00], stream id[4] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.452 [api_impl.cc:1228] 619423 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.430.466 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.430.481 [single_op_manager.cc:50]619423 ReleaseResource:ReleaseResource in. resource id = 0xaaab02f1dfc0 [INFO] GE(619423,python3.9):2025-04-03-09:35:52.430.498 [caching_allocator.cc:402]619423 TryFreeBlocks:Try free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.430.509 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:52.430.520 [npu_memory_allocator.cc:72]619423 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.430.534 [stream_executor.cpp:197]619423 RemoveExecutor: To remove executor by stream = 187651465535424 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.430.548 [acl_resource_manager.cpp:823]619423 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.560 [stream.cc:852] 619423 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.576 [stream.cc:938] 619423 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.590 [stream.cc:1507] 619423 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.612 [stream_sqcq_manage.cc:451] 619423 FreeLogicCq: Return(0), threadIdentifier(281473725853728), devId(1), tsId(0), cqId(412), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.633 [npu_driver.cc:6163] 619423 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=4, logicCq=412. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.430.666 [npu_driver.cc:1180] 619423 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=412, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.433.745 [stream.cc:237] 619423 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.433.770 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.433.786 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.434.284 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.434.305 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.434.335 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.434.352 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=4, device_id=1, tsId=0, streamFlag=1536 [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.007 [stream.cc:185] [619423][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab02f1dfc0], stream id[4] [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.435.028 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.045 [adapter_rts.cc:321] [619423][hrtCtxSetCurrent] success [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.058 [stream.cc:193] [619423]Restore Ctx ret[0], setCtx[0xaaaaf7bf5f00], stream id[4] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.089 [hcom_executor.cc:45] [619423][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.435.125 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.435.151 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.166 [hcom_ops_kernel_info_store.cc:3423] [619423]finalize hccl kernel info store. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.182 [hcom_common.cc:85] [619423][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[1] [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.204 [plugin_manager.cc:56] [619423]hccl ops plugin finalize start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.219 [hcom_plugin.cc:90] [619423]Finalize start. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.314 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.337 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.351 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.368 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.380 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.395 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.408 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.418 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.450 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.466 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.477 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.490 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.525 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.540 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.556 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.567 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.611 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.623 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.635 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.649 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.676 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.687 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.700 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.714 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.743 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.754 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.765 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.778 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.804 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.816 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.824 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.834 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.864 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.877 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.887 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.897 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.930 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.942 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.954 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.966 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.435.998 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.010 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.020 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.032 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.066 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.080 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.089 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.105 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.133 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.148 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.183 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.205 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.235 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.247 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.261 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.272 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.300 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.311 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.324 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.338 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.368 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.380 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.392 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.407 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.419 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.431 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.443 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.459 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.477 [hcom_plugin.cc:119] [619423]Finalize: HcomDestroy success. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.489 [plugin_manager.cc:62] [619423]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.436.558 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.436.582 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.591 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.607 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.618 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.629 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.638 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.651 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.662 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.675 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.687 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.697 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.717 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.730 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.742 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.752 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.764 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.774 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.785 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.797 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.811 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.822 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.833 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.845 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.854 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.865 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.880 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.893 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.905 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.917 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.927 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.937 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.952 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.963 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.973 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.983 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.436.996 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.010 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.023 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.034 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.043 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.052 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.062 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.075 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.090 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.101 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.123 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.135 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.146 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.160 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.169 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.180 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.190 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.201 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.212 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.224 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.235 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.249 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.260 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.272 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.283 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.293 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.304 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.314 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.327 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.337 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.347 [hcom_common.cc:958] [619423][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.359 [hcom_common.cc:961] [619423]Entry-HcomDestroy:void skip [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.370 [hcom.cc:337] [619423][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:52.437.387 [hcom.cc:340] [619423]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.416 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.434 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.617 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.641 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.659 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.672 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.696 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.714 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.729 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.743 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.757 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.771 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.787 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.799 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.815 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.828 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61029220352 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.842 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.857 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61029219840, total count : 1 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.871 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.896 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116404.00MB,"idle mem":116403.99MB,"used by event mem":0.00MB,"eager free mem":116403.99MB}. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.917 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.938 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.967 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.437.983 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.001 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.019 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.035 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.051 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.068 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.084 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.099 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.116 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61029220352,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.128 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.143 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.163 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffeac100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.181 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffeac101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.438.202 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58202M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.224 [callback.cpp:37]619423 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.242 [callback.cpp:44]619423 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.294 [context.cpp:72]619423 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.314 [api_impl.cc:3717] 619423 ContextSetCurrent: set context=aaaaf7bf5f00, flag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.329 [context.cpp:80]619423 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.341 [stream.cpp:75]619423 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619423,python3.9):2025-04-03-09:35:52.438.360 [stream_manage_func_registry.cc:22]619423 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.373 [stream.cpp:80]619423 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.386 [api_impl.cc:1228] 619423 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.398 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.438.410 [single_op_manager.cc:50]619423 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf8b5f1c0 [INFO] GE(619423,python3.9):2025-04-03-09:35:52.438.424 [caching_allocator.cc:402]619423 TryFreeBlocks:Try free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.438.439 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:52.438.452 [npu_memory_allocator.cc:72]619423 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.464 [stream_executor.cpp:197]619423 RemoveExecutor: To remove executor by stream = 187651293835712 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.476 [acl_resource_manager.cpp:823]619423 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.488 [stream.cc:852] 619423 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.504 [stream.cc:938] 619423 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.522 [event.cc:1036] 619423 WaitForBusy: device_id=1, event_id=0 finished map already empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.539 [npu_driver.cc:626] 619423 EventIdFree: event id free begin, device_id=1, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.562 [raw_device.cc:1158] 619423 FreeEventIdFromDrv: device_id=1, event_id=0 tear down. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.577 [event.cc:61] 619423 ~Event: device_id=1, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.595 [npu_driver.cc:3803] 619423 SqCqFree: deviceId=1, tsId=0, sqId=133, cqId=133 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.631 [subscribe.cc:244] 619423 Delete: delete callback subscribe: groupId=320, threadId=281471981645728, devId=1, tsId=0, streamId=2, sqId=133, cqId=133 [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.438.631 [trs_sqcq.c:1491][ascend][curpid:619423,619598][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.650 [stream.cc:1507] 619423 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.656 [npu_driver.cc:3833] 619598 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.671 [api_impl.cc:5058] 619598 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.673 [stream_sqcq_manage.cc:451] 619423 FreeLogicCq: Return(0), threadIdentifier(281473725853728), devId(1), tsId(0), cqId(410), isFastCq(0). [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.685 [callback.cpp:132]619598 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.703 [npu_driver.cc:6163] 619423 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=2, logicCq=410. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.710 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.727 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.738 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.745 [npu_driver.cc:1180] 619423 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=410, drvFlag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.752 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.765 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.778 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.792 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.802 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.812 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.825 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.835 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.847 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.857 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.870 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.884 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.899 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.911 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.923 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.933 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.944 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.438.956 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.983 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.438.994 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.003 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.015 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.030 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.040 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.054 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.066 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.080 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.092 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.103 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.113 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.124 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.135 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.148 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.158 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.170 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.183 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.195 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.208 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.222 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.236 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.247 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.259 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.272 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.291 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.303 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.314 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.326 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.337 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.350 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.362 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.373 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.382 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.396 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.409 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.422 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.435 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.447 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.462 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.476 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.489 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.499 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.509 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.520 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.535 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.548 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.559 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.570 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.584 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.605 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.617 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.630 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.639 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.650 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.659 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.677 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.689 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.704 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.715 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.729 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.740 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.752 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.764 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.775 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.790 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.804 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.818 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.835 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.849 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.863 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.874 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.884 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.897 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.908 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.924 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.936 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.946 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.959 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.439.975 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.439.992 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.003 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.013 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.024 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.037 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.049 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.061 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.071 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.083 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.093 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.104 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.114 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.128 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.139 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.153 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.188 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.203 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.219 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.231 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.252 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.265 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.276 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.288 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.300 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.312 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.324 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.334 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.347 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.357 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.370 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.380 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.392 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.402 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.413 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.423 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.435 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.451 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.462 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.472 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.486 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.497 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.510 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.522 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.532 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.553 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.564 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.575 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.589 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.605 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.618 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.632 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.643 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.653 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.664 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.676 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.689 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.703 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.717 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.728 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.739 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.753 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.764 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.774 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.784 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.796 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.808 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.819 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.832 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.846 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.865 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.876 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.887 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.900 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.912 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.923 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.934 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.947 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.958 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.970 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.440.983 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.440.993 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.003 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.015 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.027 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.040 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.054 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.069 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.081 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.093 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.103 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.113 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.125 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.138 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.151 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.172 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.182 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.192 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.205 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.219 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.231 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.245 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.256 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.266 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.278 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.292 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.303 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.316 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.327 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.338 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.347 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.358 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.372 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.386 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.398 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.408 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.417 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.430 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.443 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.456 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.475 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.486 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.499 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.512 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.526 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.537 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.547 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.558 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.569 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.579 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.590 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.602 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.616 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.629 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.642 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.651 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.662 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.675 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.685 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.695 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.705 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.719 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.733 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.747 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.767 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.777 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.789 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.801 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.813 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.826 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.835 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.844 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.855 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.868 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.883 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.895 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.905 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.914 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.927 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.942 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.954 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.965 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.441.976 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.441.990 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.000 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.010 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.023 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.035 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.044 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.064 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.079 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.094 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.108 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.120 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.131 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.142 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.152 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.166 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.178 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.188 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.198 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.209 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.220 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.232 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.242 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.252 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.263 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.276 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.286 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.296 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.307 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.319 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.330 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.341 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.362 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.373 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.383 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.393 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.402 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.415 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.427 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.442 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.456 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.467 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.478 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.488 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.499 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.512 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.523 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.536 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.547 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.559 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.572 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.584 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.595 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.605 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.616 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.628 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.639 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.665 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.679 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.691 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.702 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.716 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.729 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.743 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.754 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.763 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.775 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.789 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.800 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.812 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.821 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.833 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.845 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.855 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.865 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.877 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.889 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.903 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.916 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.928 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.938 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.957 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.442.971 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.984 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.442.994 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.004 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.013 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.027 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.039 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.051 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.060 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.072 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.085 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.098 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.109 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.121 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.133 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.144 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.157 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.169 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.181 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.191 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.205 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.219 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.232 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.240 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.257 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.269 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.284 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.295 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.306 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.317 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.330 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.338 [stream.cc:237] 619423 FreeStreamId: Free stream_id=2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.344 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.356 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.361 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.371 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=1, tsId=0, drvFlag=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.378 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.392 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.404 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.415 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.429 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.443 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.455 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.465 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.474 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.489 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.499 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.509 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.519 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.531 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.552 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.565 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.575 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.585 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.596 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.610 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.623 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.633 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.644 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.657 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.667 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.683 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.695 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.709 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.721 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.734 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.746 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.756 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.768 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.778 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.790 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.802 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.815 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.827 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.848 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.862 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.874 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.885 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.888 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=1, tsId=0, drvFlag=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.896 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.905 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.911 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.924 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=2, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.928 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.940 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=2, device_id=1, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.942 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.959 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.443.973 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.987 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.443.998 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.010 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.021 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.031 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.041 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.054 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.066 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.078 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.087 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.100 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.119 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.131 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.140 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.150 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.173 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.188 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.202 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.213 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.224 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.237 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.247 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.257 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.267 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.278 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.290 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.305 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.316 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.325 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.335 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.350 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.363 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.375 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.386 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.398 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.412 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.423 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.432 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.446 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.458 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.469 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.478 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.489 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.500 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.510 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.520 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.531 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.542 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.553 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.564 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.573 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.585 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.598 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.607 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.617 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.627 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.638 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.649 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.660 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.670 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.691 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.703 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.712 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.726 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.738 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.750 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.759 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.769 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.781 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.791 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.800 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.812 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.824 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.836 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.847 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.858 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.870 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.884 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.898 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.907 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.917 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.928 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.941 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.951 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.961 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.444.985 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.444.998 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.007 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.016 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.029 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.041 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.049 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.061 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.073 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.082 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.091 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.101 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.115 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.128 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.138 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.148 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.161 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.172 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.182 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.193 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.205 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.216 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.228 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.243 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.255 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.273 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.285 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.296 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.306 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.316 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.329 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.341 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.352 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.364 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.378 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.387 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.396 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.407 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.418 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.429 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.440 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.448 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.460 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.473 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.485 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.496 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.508 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.519 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.529 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.550 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.560 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.573 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.588 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.602 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.612 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.627 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.643 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.653 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.664 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.677 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.686 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.696 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.709 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.720 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.730 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.740 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.752 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.763 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.773 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.785 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.795 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.805 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.815 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.826 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.845 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.856 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.866 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.877 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.885 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.900 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.912 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.924 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.935 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.949 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.962 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.975 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.445.987 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.445.997 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.013 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.028 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.039 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.051 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.062 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.073 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.085 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.097 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.106 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.117 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.128 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.145 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.156 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.166 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.175 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.188 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.200 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.211 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.223 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.235 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.244 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.254 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.266 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.275 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.286 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.298 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.314 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.325 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.338 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.347 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.356 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.367 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.377 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.390 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.402 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.412 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.438 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.450 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.461 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.465 [stream.cpp:87]619423 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.476 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.490 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.501 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.446.508 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaaf8b5f1c0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.514 [callback.cpp:116]619598 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.526 [stream.cpp:75]619423 aclrtDestroyStream: start to execute aclrtDestroyStream [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.530 [subscribe.cc:308] 619598 GetGroupIdByThreadId: can not find subscribe info, threadId=281471981645728 [INFO] GE(619423,python3.9):2025-04-03-09:35:52.446.541 [stream_manage_func_registry.cc:22]619423 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.543 [api_impl.cc:5043] 619598 ProcessReport: get groupId fail, threadIdentifier=281471981645728, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.557 [stream.cpp:80]619423 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.563 [callback.cpp:130]619598 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.571 [api_impl.cc:1228] 619423 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.586 [acl_resource_manager.cpp:814]619423 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.446.600 [single_op_manager.cc:50]619423 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf8c90f80 [INFO] GE(619423,python3.9):2025-04-03-09:35:52.446.615 [caching_allocator.cc:402]619423 TryFreeBlocks:Try free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.446.627 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] DEVICE(619423,ffff4d7aefa0,python3.9):2025-04-03-09:35:52.446.628 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] GE(619423,python3.9):2025-04-03-09:35:52.446.643 [npu_memory_allocator.cc:72]619423 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.655 [stream_executor.cpp:197]619423 RemoveExecutor: To remove executor by stream = 187651295088512 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.668 [acl_resource_manager.cpp:823]619423 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.681 [stream.cc:852] 619423 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.707 [stream.cc:938] 619423 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.720 [event.cc:1036] 619423 WaitForBusy: device_id=1, event_id=1 finished map already empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.735 [npu_driver.cc:626] 619423 EventIdFree: event id free begin, device_id=1, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.756 [raw_device.cc:1158] 619423 FreeEventIdFromDrv: device_id=1, event_id=1 tear down. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.769 [event.cc:61] 619423 ~Event: device_id=1, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.784 [npu_driver.cc:3803] 619423 SqCqFree: deviceId=1, tsId=0, sqId=134, cqId=134 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.816 [subscribe.cc:244] 619423 Delete: delete callback subscribe: groupId=321, threadId=281471973191584, devId=1, tsId=0, streamId=3, sqId=134, cqId=134 [WARNING] DRV(619423,python3.9):2025-04-03-09:35:52.446.823 [trs_sqcq.c:1491][ascend][curpid:619423,619599][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=1; type=1; ret=16) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.832 [stream.cc:1507] 619423 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.853 [npu_driver.cc:3833] 619599 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=1, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.869 [api_impl.cc:5058] 619599 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.872 [stream_sqcq_manage.cc:451] 619423 FreeLogicCq: Return(0), threadIdentifier(281473725853728), devId(1), tsId(0), cqId(411), isFastCq(0). [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.888 [callback.cpp:132]619599 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.893 [npu_driver.cc:6163] 619423 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=3, logicCq=411. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.904 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.922 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.926 [npu_driver.cc:1180] 619423 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=411, drvFlag=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.933 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.945 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.956 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.968 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.446.978 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.446.988 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.001 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.011 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.042 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.055 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.068 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.081 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.091 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.102 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.114 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.127 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.137 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.148 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.157 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.167 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.179 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.189 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.198 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.212 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.223 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.233 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.245 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.255 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.265 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.277 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.287 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.297 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.306 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.330 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.344 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.355 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.365 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.379 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.390 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.402 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.416 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.425 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.434 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.447 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.460 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.471 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.482 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.495 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.506 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.516 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.525 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.537 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.550 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.563 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.575 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.589 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.602 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.612 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.635 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.646 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.655 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.667 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.679 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.689 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.700 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.713 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.726 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.736 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.746 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.759 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.769 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.780 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.790 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.800 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.811 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.825 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.837 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.848 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.860 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.872 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.881 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.891 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.904 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.921 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.933 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.946 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.956 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.967 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.447.980 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.447.991 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.005 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.017 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.028 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.039 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.049 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.061 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.072 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.082 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.093 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.108 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.121 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.132 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.144 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.154 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.177 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.187 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.199 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.221 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.232 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.243 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.257 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.269 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.278 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.291 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.303 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.313 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.325 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.340 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.350 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.364 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.376 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.386 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.397 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.410 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.423 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.433 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.447 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.458 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.468 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.479 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.491 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.504 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.520 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.533 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.545 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.556 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.568 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.581 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.593 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.606 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.619 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.632 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.644 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.658 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.679 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.690 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.700 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.711 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.719 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.729 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.738 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.747 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.760 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.771 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.781 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.791 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.802 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.820 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.830 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.839 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.849 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.860 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.870 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.882 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.892 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.902 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.911 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.922 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.932 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.942 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.952 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.963 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.448.973 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.983 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.448.991 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.002 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.011 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.022 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.032 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.042 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.052 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.062 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.081 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.092 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.103 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.112 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.126 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.137 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.147 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.158 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.168 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.178 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.188 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.199 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.208 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.219 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.223 [stream.cc:237] 619423 FreeStreamId: Free stream_id=3. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.230 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.237 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.241 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.249 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=1, tsId=0, drvFlag=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.252 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.265 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.274 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.285 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.295 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.305 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.320 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.331 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.341 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.351 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.361 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.371 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.381 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.392 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.401 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.411 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.421 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.432 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.441 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.451 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.463 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.473 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.483 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.493 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.503 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.512 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.521 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.531 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.542 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.553 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.562 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.577 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.587 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.597 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.606 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.617 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.627 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.638 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.648 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.658 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.668 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.678 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.688 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.698 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.708 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.718 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.728 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.739 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.749 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.759 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.769 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.779 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.792 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.803 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.813 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.824 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.825 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=1, tsId=0, drvFlag=0 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.841 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.847 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.852 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.862 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.864 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.877 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=3, device_id=1, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.878 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.894 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.906 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.916 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.927 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.937 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.947 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.957 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.967 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.449.977 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.988 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.449.999 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.010 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.020 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.031 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.041 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.052 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.067 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.078 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.087 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.097 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.108 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.118 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.128 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.139 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.149 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.158 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.169 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.179 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.189 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.200 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.209 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.220 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.231 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.240 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.250 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.262 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.273 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.283 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.295 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.305 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.316 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.331 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.341 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.351 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.361 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.372 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.381 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.391 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.402 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.413 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.421 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.432 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.442 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.452 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.461 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.472 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.481 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.481 [event.cc:61] 619423 ~Event: device_id=1, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.492 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.501 [stream.cpp:87]619423 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.503 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.517 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.526 [callback.cpp:116]619599 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.532 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaaf8c90f80. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.536 [subscribe.cc:308] 619599 GetGroupIdByThreadId: can not find subscribe info, threadId=281471973191584 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.551 [device.cpp:183]619423 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 1. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.555 [api_impl.cc:5043] 619599 ProcessReport: get groupId fail, threadIdentifier=281471973191584, reCode=0x7160004 [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.568 [callback.cpp:130]619599 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.569 [api_impl.cc:3419] 619423 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.584 [runtime.cc:3444] 619423 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.450.597 [api_impl.cc:3449] 619423 DeviceReset: Succ, userDeviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.608 [device.cpp:189]619423 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 1. [INFO] DEVICE(619423,ffff4cf9efa0,python3.9):2025-04-03-09:35:52.450.621 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.632 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.670 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.700 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.718 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.731 [callback.cpp:37]619423 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:52.450.747 [callback.cpp:44]619423 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.768 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:52.450.791 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.498.749 [awatchdog_monitor.c:160](tid:619461) thread 619667 not exist [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.498.787 [awatchdog_monitor.c:160](tid:619461) thread 619668 not exist [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.498.802 [awatchdog_monitor.c:160](tid:619461) thread 619669 not exist [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:52.498.814 [awatchdog_monitor.c:160](tid:619461) thread 619670 not exist [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.748.716 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.748.749 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.748.785 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:52.748.798 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.748.816 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:52.748.882 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.024.181 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.024.181 [queue_client_comm.c:114][ascend][curpid:619423,619655][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.306 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.024.327 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.331 [npu_driver.cc:4338] 619655 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.024.343 [tensor_data_transfer.cpp:1054]619663 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 1, name is ms_scalar_summary [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.024.348 [tensor_data_transfer.cpp:612]619655 acltdtReceiveTensorV2: queue is empty, device is 1, name is _npu_log [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.024.358 [tensor_data_transfer.cpp:1056]619663 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.370 [api_impl.cc:6121] 619663 MemQueuePeek: Start to peek queue on devId 1, qid is 4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.382 [npu_driver.cc:4330] 619663 MemQueuePeek: peek mem queue, deviceId=1, qid=4, timeout=800. [INFO] DEVICE(619423,ffff13734fa0,python3.9):2025-04-03-09:35:53.024.479 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.024.615 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is _npu_log [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.642 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.024.655 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=0. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.025.091 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is _npu_log [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.025.129 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.025.151 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.028.181 [queue_client_comm.c:114][ascend][curpid:619423,619665][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.028.181 [queue_client_comm.c:114][ascend][curpid:619423,619657][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.028.187 [queue_client_comm.c:114][ascend][curpid:619423,619659][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.028.188 [queue_client_comm.c:114][ascend][curpid:619423,619661][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.272 [npu_driver.cc:4338] 619665 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.028.290 [tensor_data_transfer.cpp:612]619665 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_histogram_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.296 [npu_driver.cc:4338] 619657 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.028.312 [tensor_data_transfer.cpp:612]619657 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_dump [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.326 [npu_driver.cc:4338] 619659 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.028.341 [tensor_data_transfer.cpp:612]619659 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_tensor_summary [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.356 [npu_driver.cc:4338] 619661 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.028.371 [tensor_data_transfer.cpp:612]619661 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_image_summary [INFO] DEVICE(619423,fffee37eefa0,python3.9):2025-04-03-09:35:53.028.437 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619423,fffefefdefa0,python3.9):2025-04-03-09:35:53.028.456 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619423,fffeffffefa0,python3.9):2025-04-03-09:35:53.028.445 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619423,fffefdfbefa0,python3.9):2025-04-03-09:35:53.028.480 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.028.682 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is ms_histogram_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.704 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.028.717 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=5. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.029.150 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is ms_histogram_summary [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.029.181 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.029.203 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [WARNING] DRV(619423,python3.9):2025-04-03-09:35:53.856.202 [queue_client_comm.c:114][ascend][curpid:619423,619663][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:53.856.393 [npu_driver.cc:4338] 619663 MemQueuePeek: queue empty [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.856.425 [tensor_data_transfer.cpp:612]619663 acltdtReceiveTensorV2: queue is empty, device is 1, name is ms_scalar_summary [INFO] DEVICE(619423,fffefcf9efa0,python3.9):2025-04-03-09:35:53.856.649 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.856.900 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is ms_scalar_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.856.950 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 4. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.856.968 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=4. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.857.372 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is ms_scalar_summary [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.857.429 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.857.464 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.857.524 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is ms_image_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.857.544 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 3. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.857.556 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=3. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.857.961 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is ms_image_summary [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.857.991 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.858.013 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.858.087 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is ms_tensor_summary [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.858.107 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.858.121 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=2. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.858.476 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is ms_tensor_summary [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.858.505 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.858.526 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.858.576 [tensor_data_transfer.cpp:973]619423 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 1, name is ms_tensor_dump [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.858.595 [api_impl.cc:6065] 619423 MemQueueDestroy: Start to destroy queue on devId 1, qid is 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.858.607 [npu_driver.cc:3989] 619423 MemQueueDestroy: deviceId=1, qid=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.858.956 [tensor_data_transfer.cpp:977]619423 acltdtDestroyChannel: acltdtDestroyChannel success, device is 1, name is ms_tensor_dump [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.859.001 [device.cpp:183]619423 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.859.026 [api_impl.cc:3419] 619423 DeviceReset: reset device, device_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.859.040 [runtime.cc:3444] 619423 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.859.057 [api_impl.cc:3449] 619423 DeviceReset: Succ, userDeviceId=1. [INFO] ASCENDCL(619423,python3.9):2025-04-03-09:35:53.859.068 [device.cpp:189]619423 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 1. [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.094 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.127 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.173 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.193 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.215 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_1 [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.233 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.256 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.273 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.859.302 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.544 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.595 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.616 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.634 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.664 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.695 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.721 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.739 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] PS(619423,ffff4e7cefa0,python3.9):2025-04-03-09:35:53.859.767 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] DISTRIBUTED(619423,fffeff7eefa0,python3.9):2025-04-03-09:35:53.859.780 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] PS(619423,ffff4dfbefa0,python3.9):2025-04-03-09:35:53.859.780 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619423,fffebfffefa0,python3.9):2025-04-03-09:35:53.859.879 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.128 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.151 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.178 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.204 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.222 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.237 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.253 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.427 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.461 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.514 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.534 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.549 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.568 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.583 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.619 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.636 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.653 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.669 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.722 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.860.740 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.146 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.178 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.215 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.234 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.249 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.267 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619423,ffffb5717020,python3.9):2025-04-03-09:35:53.861.612 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.964.759 [runtime.cc:5239] 619592 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:53.964.828 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:53.964.850 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619423,python3.9):2025-04-03-09:35:53.964.861 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619592] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.964.887 [runtime.cc:3077] 619592 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=1, devId=1, tdt status=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:53.965.017 [npu_driver.cc:551] 619592 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:54.028.290 [awatchdog_monitor.c:160](tid:619461) thread 619671 not exist [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:54.291.140 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619423,ffffb5717020,python3.9):2025-04-03-09:35:54.291.296 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.631.827 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.315 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.355 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.868 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.892 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.908 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.925 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.950 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.632.966 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.633.959 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.633.984 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.090 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.139 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.289 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.308 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.321 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.335 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.350 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.363 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.376 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.389 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.403 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.415 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.430 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.443 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.456 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.468 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.481 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.491 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.502 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.515 [heartbeat.cc:145] [619423][DeInit]heartbeat deinit begin [INFO] HCCL(619423,python3.9):2025-04-03-09:35:54.634.529 [heartbeat.cc:183] [619423][DeInit]heartbeat deinit end [INFO] DEVICE(619423,ffffb5717020,python3.9):2025-04-03-09:35:54.634.826 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 1, group name: hccl_world_group [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:54.634.936 [hash_data.cpp:63] >>> (tid:619423) HashData uninitialize success [INFO] PROFILING(619423,python3.9):2025-04-03-09:35:54.634.962 [msprof_reporter_mgr.cpp:270] >>> (tid:619423) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.638.070 [api_impl.cc:2003] 619423 DevFree: device free mem=0x12c041200000 [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:54.639.464 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619423,ffffb5717020,python3.9):2025-04-03-09:35:54.639.497 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:54.639.643 [adx_dump_record.cpp:229][tid:619423] start to dump uninit [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:54.639.676 [adx_dump_record.cpp:239][tid:619423] dump uninit success [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:54.639.697 [adx_dump_record.cpp:674][tid:619653] exit record file thread [INFO] GE(619423,python3.9):2025-04-03-09:35:54.704.387 [operator_factory_impl.cc:392]619423 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.570 [operator_factory_impl.cc:392]619423 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.623 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.684 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.700 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.714 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.726 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.739 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.751 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.719.762 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.878.819 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.879.028 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] GE(619423,python3.9):2025-04-03-09:35:54.879.082 [opsproto_manager.cc:66]619423 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.934.691 [api_impl.cc:7151] 619423 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.934.922 [api_impl.cc:7135] 619423 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.937.265 [api_impl.cc:7151] 619423 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.937.462 [api_impl.cc:7135] 619423 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.938.913 [api_impl.cc:7121] 619423 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.942.563 [api_impl.cc:7121] 619423 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:54.948.980 [osal.cc:321] 619423 ThreadExit: ThreadExit start. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:55.180.740 [stars_engine.cc:1833] 619592 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:55.180.791 [stars_engine.cc:1836] 619592 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619423,ffffb5717020,python3.9):2025-04-03-09:35:55.181.073 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619423,python3.9):2025-04-03-09:35:55.212.673 [ops_kernel_builder_registry.cc:71]619423 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619423,python3.9):2025-04-03-09:35:55.212.733 [ops_kernel_builder_registry.cc:41]619423 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:56.484.202 [adx_server_manager.cpp:236][tid:619654] server manager stop [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:56.667.297 [adx_hdc_epoll.cpp:57][tid:619423] start to close hdc epoll [INFO] GE(619423,python3.9):2025-04-03-09:35:56.775.807 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafd7c3b70]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.777.801 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafd7c81a0]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.780.559 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafd7f2620]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.784.176 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafd7c51b0]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.785.131 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafebfe7a0]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.786.069 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafec26790]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.787.359 [op_impl_registry_holder_manager.cc:31]619423 CloseHandle:start close handle, handle[0xaaaafece9e10]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.807.473 [mem_manager.cc:118]619423 Finalize:MemManager Finalize. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.807.512 [session_scope_mem_allocator.cc:73]619423 FreeAllMemory:Free all memory [INFO] GE(619423,python3.9):2025-04-03-09:35:56.809.917 [session_scope_mem_allocator.cc:73]619423 FreeAllMemory:Free all memory [INFO] GE(619423,python3.9):2025-04-03-09:35:56.809.950 [caching_allocator.cc:474]619423 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.809.985 [caching_allocator.cc:391]619423 FreeBlocks:Free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.003 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.015 [caching_allocator.cc:422]619423 FreeBlockBins:Free block bins. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.028 [caching_allocator.cc:474]619423 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.040 [caching_allocator.cc:391]619423 FreeBlocks:Free blocks. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.053 [caching_allocator.cc:358]619423 FreeCachedBlocks:Free cached blocks [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.065 [caching_allocator.cc:422]619423 FreeBlockBins:Free block bins. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.087 [mem_manager.cc:145]619423 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.107 [rts_caching_mem_allocator.cc:150]619423 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619423,python3.9):2025-04-03-09:35:56.810.174 [rts_caching_mem_allocator.cc:136]619423 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.065 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.141 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.162 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.175 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.188 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.200 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.233 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.249 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.259 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.270 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.282 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.293 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.305 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.317 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.333 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] HCCL(619423,python3.9):2025-04-03-09:35:56.917.344 [network_manager.cc:875] [619423]destroy all nic/vnic. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.503 [runtime.cc:1570] 619423 ~Runtime: deconstruct runtime [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.544 [runtime.cc:1577] 619423 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.830 [raw_device.cc:262] 619423 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.866 [engine.cc:188] 619423 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.893 [ringbuffer_maintain_task.cc:83] 619423 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.912 [stream.cc:1400] 619423 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.927 [stream.cc:2595] 619423 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=1 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.955 [npu_driver.cc:751] 619423 SqTaskSend: device_id=1, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.973 [stars_engine.cc:893] 619423 SendTask: Sq task send finished, device_id=1, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.923.991 [stream.cc:1494] 619423 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.013 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.031 [stream.cc:2886] 619423 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.046 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.064 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.080 [task.cc:185] 619423 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.098 [logger.cc:152] 619423 TaskFinished: device_id=1, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.122 [stars_engine.cc:437] 619423 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.139 [stream.cc:2886] 619423 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.155 [stars_engine.cc:524] 619423 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.185 [npu_driver.cc:5547] 619423 GetSqHead: deviceId=1, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.206 [stream.cc:852] 619423 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.225 [stream.cc:938] 619423 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.924.238 [stream.cc:1507] 619423 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.927.868 [stream_sqcq_manage.cc:451] 619423 FreeLogicCq: Return(0), threadIdentifier(281473725853728), devId(1), tsId(0), cqId(409), isFastCq(0). [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.927.917 [npu_driver.cc:6163] 619423 StreamUnBindLogicCq: success: deviceId=1, tsId=0, streamId=0, logicCq=409. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.927.972 [npu_driver.cc:1180] 619423 LogicCqFree: Succ:device_id=1,ts_id=0,type=2,logic_cq_id=409, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.927.994 [stream.cc:237] 619423 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.009 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.040 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.679 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.702 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.722 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.744 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=0, device_id=1, tsId=0, streamFlag=192 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.812 [stars_engine.cc:206] 619593 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.962 [stars_engine.cc:191] 619423 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.981 [osal.cc:225] 619423 Join: mmJoinTask failed, thread=281472096460704 does not exist! [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.928.994 [stars_engine.cc:150] 619423 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.014 [runtime.cc:5239] 619423 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=1, deviceId=1. [INFO] TDT(619423,python3.9):2025-04-03-09:35:56.929.035 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619423] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.053 [runtime.cc:3003] 619423 StopAicpuExecutor: TsdClose success, userDeviceId=1, devId=1, tdt status=0 [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:56.929.072 [atrace_api.c:118](tid:619423) AtraceDestroy start. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:56.929.096 [tracer_mgr_operate.c:248](tid:619423) destroy object RUNTIME_ATRACE_DEV1_TS0, exitSave(false). [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:56.929.116 [atrace_api.c:120](tid:619423) AtraceDestroy end. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.134 [stream.cc:237] 619423 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.151 [stream_sqcq_manage.cc:216] 619423 DeAllocStreamSqCq: deviceId=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.162 [npu_driver.cc:966] 619423 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.373 [npu_driver.cc:974] 619423 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=1, tsId=0, drvFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.392 [stream_sqcq_manage.cc:228] 619423 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.405 [npu_driver.cc:432] 619423 StreamIdFree: stream id free begin, device_id=1, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.929.422 [npu_driver.cc:441] 619423 StreamIdFree: Stream id free success, stream_id=1, device_id=1, tsId=0, streamFlag=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.943.755 [stars_engine.cc:58] 619423 ~StarsEngine: Destructor. [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:56.943.790 [engine.cc:84] 619423 ~Engine: Destructor. [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.652 [trs_dev_drv.c:370][ascend][curpid:619423,619423][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=1; fd=36; mmapFd=37) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.683 [devmm_svm.c:2238][ascend][curpid:619423,619423][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.725 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.747 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.766 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.784 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.801 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev1 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.821 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev1 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.842 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev1 Cached_size:1961984Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.855 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev1 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.874 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev1 Cached_size:2097152Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.888 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev1 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.907 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev1 Cached_size:2097152Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.922 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev1 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.941 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev1 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.958 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev1 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.975 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev1 Cached_size:23068672Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.944.992 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev1 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.945.017 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev1 Cached_size:6291456Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.945.030 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev1 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.945.047 [svm_mem_statistics.c:140][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev1 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619423,python3.9):2025-04-03-09:35:56.945.067 [svm_mem_statistics.c:146][ascend][curpid:619423,619423][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev1 Cached_size:16777216Bytes [INFO] DRV(619423,python3.9):2025-04-03-09:35:57.016.863 [drv_log_user.c:623][ascend][curpid:619423,619423][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=1; phyid=1; vfid=0; hostpid=619423; devpid=31531) [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:57.016.930 [npu_driver.cc:3045] 619423 DeviceClose: Close device success, device_id=1, tsId=0 [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:57.016.954 [raw_device.cc:125] 619423 ~Device: deconstruct device [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:57.128.803 [plugin_old_arch.cc:25] 619423 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:57.128.873 [runtime_keeper.cc:167] 619423 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619423,python3.9):2025-04-03-09:35:57.129.884 [task_fail_callback_data_manager.cc:63] 619423 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:57.230.634 [stacktrace_signal.c:188](tid:619423) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:57.305.893 [atrace_client_communication.c:141](tid:619423) send [end] msg. [INFO] DRV(619423,python3.9):2025-04-03-09:35:57.307.914 [hdc_core.c:1982][ascend][curpid:619423,619423][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4282; pid=619423) [INFO] IDEDD(619423,python3.9):2025-04-03-09:35:57.307.951 [hdc_api.cpp:671][tid:619423] connect succ, peer_node: 0, peer_devid: 1 [INFO] DRV(619423,python3.9):2025-04-03-09:35:57.308.020 [hdc_client.c:417][ascend][curpid:619423,619423][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4282) [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:57.308.078 [atrace_client_communication.c:164](tid:619423) send [end] msg success. [INFO] DRV(619423,python3.9):2025-04-03-09:35:58.382.248 [hdc_client.c:417][ascend][curpid:619423,619575][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4232) [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:58.382.312 [atrace_client_core.c:127](tid:619575) atrace receive thread exited, devId=1. [INFO] ATRACE(619423,python3.9):2025-04-03-09:35:58.383.102 [awatchdog_monitor.c:234](tid:619423) AwdMonitorExit [INFO] DRV(619423,python3.9):2025-04-03-09:35:58.402.627 [uda_user.c:147][ascend][curpid:619423,619423][drv][devmng][uda_char_dev_open]The fd and pid info: (fd=3; pid=619423) [INFO] DRV(619423,python3.9):2025-04-03-09:35:58.402.781 [uda_user.c:317][ascend][curpid:619423,619423][drv][devmng][_uda_init]User info. (admin_flag=1; local_flag=0; max_dev_num=100; max_udev_num=1124; uda_dev_num_davinci=8; uda_dev_num_kunpeng=0) output_allreduce/worker_7.log0000644000175100017500000667356714773362776017160 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.046.112 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.059.715 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 7 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.680 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.767 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.796 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.814 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.829 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.843 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.861 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.876 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.890 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.905 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.917 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.931 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.944 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.956 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.975 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.482.989 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.001 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.014 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.028 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.040 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.053 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.080 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.093 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.106 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.119 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.131 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.154 [ir_data_type_symbol_store.cc:240]619447 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.169 [ir_data_type_symbol_store.cc:240]619447 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.183 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.197 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.211 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.225 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.237 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.250 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.262 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.273 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.286 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.298 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.311 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.324 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.338 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.350 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.361 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.379 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.391 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.404 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.417 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.430 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.442 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.454 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.466 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.479 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.490 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.502 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.515 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.526 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.540 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.552 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.564 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.577 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.588 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.599 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.612 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.623 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.635 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.650 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.661 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.673 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.690 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.701 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.713 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.727 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.740 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.750 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.763 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.775 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.788 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.799 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.811 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.823 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.835 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.848 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.860 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.873 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.885 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.895 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.908 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.920 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.932 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.943 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.956 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.972 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.984 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.483.995 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.011 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.023 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.035 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.048 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.059 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.070 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.082 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.091 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.105 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.118 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.130 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.140 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.152 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.176 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.187 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.198 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.209 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.222 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.237 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.249 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.261 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.273 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.288 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.299 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.309 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.327 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.337 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.352 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.364 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.375 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.385 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.396 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.408 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.420 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.430 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.442 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.456 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.468 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.480 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.493 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.505 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.517 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.528 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.539 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.550 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.562 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.575 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.586 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.599 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.613 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.625 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.642 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.654 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.666 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.679 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.691 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.703 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.713 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.724 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.739 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.751 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.765 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.777 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.794 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.809 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.821 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.837 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.850 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.862 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.871 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.882 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.894 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.906 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.919 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.931 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.950 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.962 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.975 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.484.986 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.000 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.010 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.026 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.039 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.050 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.065 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.078 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.090 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.099 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.116 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.139 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.152 [ir_data_type_symbol_store.cc:173]619447 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.162 [ir_data_type_symbol_store.cc:219]619447 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619447,python3.9):2025-04-03-09:35:27.485.179 [ir_data_type_symbol_store.cc:182]619447 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.133 [awatchdog_core.c:99](tid:619447) watchdog get platform host side [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.287 [awatchdog_monitor.c:127](tid:619447) create monitor thread successfully, tid :281473035308960 [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.303 [awatchdog_monitor.c:226](tid:619447) AwdMonitorInit [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.479 [trace_attr.c:99](tid:619447) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.504 [trace_attr.c:189](tid:619447) attr init success, timeout=0ms. [WARNING] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.517 [trace_system_api.c:115](tid:619447) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.570 [trace_recorder.c:40](tid:619447) home_directory=/home/jenkins. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.583 [trace_recorder.c:121](tid:619447) use root path: /home/jenkins/ascend [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.593 [trace_event.c:151](tid:619447) init trace event success [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.491.635 [atrace_client_core.c:185](tid:619447) atrace client init success. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.492.846 [stacktrace_dumper.c:218](tid:619447) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619447,python3.9):2025-04-03-09:35:27.492.867 [trace_system_api.c:115](tid:619447) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.492.965 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 2 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.492.980 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 15 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.492.990 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 3 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.001 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 4 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.010 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 5 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.021 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 6 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.030 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 7 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.041 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 8 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.050 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 11 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.061 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 24 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.070 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 25 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.080 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 31 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.090 [stacktrace_signal.c:165](tid:619447) register signal handler for signo 35 succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.100 [stacktrace_signal.c:168](tid:619447) trace signal init succeed. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:27.493.108 [trace_core.c:51](tid:619447) trace init success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.493.701 [task_fail_callback_data_manager.cc:58] 619447 TaskFailCallBackManager: Constructor. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.502.245 [operator_factory_impl.cc:232]619447 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619447,python3.9):2025-04-03-09:35:27.502.276 [operator_factory_impl.cc:246]619447 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619447,python3.9):2025-04-03-09:35:27.502.287 [operator_factory_impl.cc:239]619447 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619447,python3.9):2025-04-03-09:35:27.508.797 [adapter_rts.cc:2720] [619447][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.525.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.525.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:27.525.130 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:27.535.867 [prof_cann_plugin.cpp:256] (tid:619447) Register module[6] callback. [INFO] OP(619447,python3.9):2025-04-03-09:35:27.586.323 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619447] cachelimit is 10000 [INFO] OP(619447,python3.9):2025-04-03-09:35:27.590.792 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619447] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619447,python3.9):2025-04-03-09:35:27.590.839 [op_dfx.cpp:54][NNOP][SystemConfig][619447] enable debug kernel flag 0. [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:27.590.854 [prof_cann_plugin.cpp:256] (tid:619447) Register module[61] callback. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.640.437 [mem_manager.cc:138]619447 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.646.866 [cache_policy.cc:47]619447 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.060 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.196 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.293 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.320 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.341 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.359 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.376 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.394 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.414 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.433 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.450 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.467 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.484 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.506 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.524 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.544 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.563 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.581 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.598 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.616 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.705 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.818 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.839 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.926 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.949 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.967 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.660.986 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.003 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.018 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.036 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.055 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.073 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.091 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.109 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.131 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.148 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.165 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.186 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.203 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.223 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.241 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.259 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.343 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.428 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.515 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.603 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.696 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.720 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.806 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.897 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.661.918 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.003 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.105 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.126 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.144 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.162 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.180 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.197 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.216 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.233 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.251 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.273 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.290 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.307 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.329 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.345 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.432 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.520 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.609 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.697 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.720 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.738 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.756 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.841 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.936 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.662.959 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.051 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.074 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.156 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.244 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.335 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.423 [addn.cc:29]619447 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.735 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.827 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619447,python3.9):2025-04-03-09:35:27.663.920 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.000 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.080 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.175 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.256 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.340 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.353 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.440 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.528 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619447,python3.9):2025-04-03-09:35:27.664.608 [symbolic_kernel_factory.cc:27]619447 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619447,python3.9):2025-04-03-09:35:27.670.616 [process_node_engine_manager.cc:140]619447 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619447,python3.9):2025-04-03-09:35:27.670.728 [process_node_engine_manager.cc:140]619447 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619447,python3.9):2025-04-03-09:35:27.676.883 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619447,python3.9):2025-04-03-09:35:27.676.921 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619447,python3.9):2025-04-03-09:35:27.677.420 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619447] start packet count timer [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:27.698.301 [prof_atls_plugin.cpp:117] (tid:619447) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:27.698.347 [prof_atls_plugin.cpp:117] (tid:619447) RegisterProfileCallback, callback type is 8 [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.773.003 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.773.540 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.821.936 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.822.096 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 7 [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.822.141 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.822.967 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.652 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.739 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.780 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.877 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.911 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.970 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.824.999 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.826.742 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.826.777 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.826.818 [acl.cpp:753]619447 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.826.892 [runtime_keeper.cc:78] 619447 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.791 [plugin_old_arch.cc:16] 619447 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaab004ebe10 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.829 [runtime_keeper.cc:149] 619447 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.841 [runtime_keeper.cc:221] 619447 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.883 [runtime.cc:1387] 619447 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.922 [runtime.cc:1163] 619447 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.949 [npu_driver.cc:4062] 619447 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.969 [runtime.cc:5096] 619447 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.983 [runtime.cc:1173] 619447 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.830.994 [runtime.cc:1183] 619447 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.015 [runtime.cc:1193] 619447 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.030 [profiler.cc:84] 619447 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.053 [runtime.cc:1209] 619447 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.083 [runtime.cc:1219] 619447 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.095 [runtime.cc:1230] 619447 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.107 [runtime.cc:1242] 619447 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.122 [runtime.cc:1264] 619447 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.135 [runtime.cc:1270] 619447 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.244 [runtime.cc:109] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.650 [runtime.cc:121] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.735 [runtime.cc:1314] 619447 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.752 [runtime.cc:1322] 619447 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.877 [runtime.cc:1338] 619447 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.937 [utils.cc:105] 619447 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.831.990 [runtime.cc:1664] 619447 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.832.031 [runtime.cc:1688] 619447 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.832.054 [runtime.cc:1953] 619447 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.832.082 [runtime.cc:1477] 619447 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:27.832.104 [prof_atls_plugin.cpp:210] (tid:619447) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:27.832.124 [heterogenous.cc:19] 619447 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:27.832.224 [api_c_soc.cc:70] 619447 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.832.284 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.832.303 [acl.cpp:760]619447 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.832.363 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.863.846 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.895.616 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.896.057 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.896.535 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.897.317 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.897.752 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.899.389 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.899.435 [acl.cpp:753]619447 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.899.462 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:27.899.474 [acl.cpp:760]619447 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:27.899.498 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.844.567 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.846.735 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.870.604 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.879.891 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.883.550 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.886.629 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.919.347 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619447,python3.9):2025-04-03-09:35:28.934.663 [platform_info.cpp:853]619447 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.873.719 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.875.522 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.875.657 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.877.936 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.878.077 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.878.198 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.878.332 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.878.444 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.879.602 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.879.740 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.881.684 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.881.819 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.881.938 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.882.055 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.882.165 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619447,python3.9):2025-04-03-09:35:29.882.275 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619447,python3.9):2025-04-03-09:35:30.501.224 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619447,python3.9):2025-04-03-09:35:34.599.334 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.854.867 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.854.953 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.854.994 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619447,python3.9):2025-04-03-09:35:34.894.406 [ops_kernel_builder_registry.cc:55]619447 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619447,python3.9):2025-04-03-09:35:34.894.500 [ops_kernel_builder_registry.cc:28]619447 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.930.966 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.931.061 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.931.141 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.931.165 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:34.931.184 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619447,python3.9):2025-04-03-09:35:35.766.758 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:37.485.732 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:37.486.354 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:37.497.123 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.594.023 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:37.594.116 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.396 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.442 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.465 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.493 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.691 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281472060592032 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.802 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281472052137888 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.828 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.595.931 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281472043683744 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.596.052 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281472035229600 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.596.080 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.596.103 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.596.292 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58722, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619447,ffff512d8fa0,python3.9):2025-04-03-09:35:37.596.321 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58722 to 127.0.0.1:8118 is successfully created. System errno: Success [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:37.596.334 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.435 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.467 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.486 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.503 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.605 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58756, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.096.627 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619447,ffff522f8fa0,python3.9):2025-04-03-09:35:38.096.635 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58756 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.596.705 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.596.729 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.596.930 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 7 has been registered successfully. [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:38.597.117 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619447,ffff43ffefa0,python3.9):2025-04-03-09:35:38.597.143 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619447,ffff43ffefa0,python3.9):2025-04-03-09:35:38.597.167 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:39.097.197 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:39.597.291 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.097.372 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(4/1200). [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.464 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3002.01 msec. [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.489 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.506 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.525 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 7 rank id: 7 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.542 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.567 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 9910 to 10165 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.591 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.082 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.618 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.645 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3002.3 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.668 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.794 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.815 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.845 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.872 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.891 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.597.909 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.599.580 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.599.604 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 9910 to 10165 [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.599.771 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:9acbce21-3597-4e13-af46-4167e90bf0fe, the ip:127.0.0.1, the port:9910 [INFO] PS(619447,ffff437eefa0,python3.9):2025-04-03-09:35:40.599.790 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619447,ffff437eefa0,python3.9):2025-04-03-09:35:40.599.817 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.601.478 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.601.620 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_7 has been registered successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.601.647 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:40.601.764 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.367 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 7 has been started successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.402 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.447 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.469 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.497 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 7, global rank size: 8 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.526 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3004.83 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.548 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 7 is devserver-c233-2 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.647 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.703 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.725 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 7 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.745 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 7 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.771 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [7]. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.808 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.254 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.871 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.902 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.602.926 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.602.974 [device.cpp:365]619447 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.079 [api_impl.cc:249] 619447 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.099 [api_impl.cc:6795] 619447 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.603.114 [device.cpp:371]619447 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.603.183 [callback.cpp:37]619447 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.603.215 [callback.cpp:44]619447 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.603.243 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.603.269 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.603.308 [device.cpp:143]619447 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.395 [npu_driver.cc:124] 619447 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.422 [api_impl.cc:2999] 619447 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.457 [runtime.cc:3602] 619447 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.603.686 [hdc_core.c:1982][ascend][curpid:619447,619447][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4240; pid=619447) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.603.813 [runtime.cc:3609] 619447 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.603.861 [atrace_client_communication.c:103](tid:619447) send [hello] msg. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.603.881 [atrace_client_communication.c:63](tid:619447) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.603.995 [hdc_core.c:1982][ascend][curpid:619447,619447][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4241; pid=619447) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:43.604.014 [hdc_api.cpp:671][tid:619447] connect succ, peer_node: 0, peer_devid: 7 [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.605.388 [hdc_client.c:417][ascend][curpid:619447,619447][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4241) [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.420 [atrace_client_communication.c:65](tid:619447) send msg by adx[finish], ret=0. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.438 [atrace_client_communication.c:97](tid:619447) msgType=1, magic=49959, version=1. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.462 [atrace_client_communication.c:135](tid:619447) send [hello] msg success. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.537 [atrace_client_core.c:160](tid:619447) start to receive device[7] trace log. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.605.568 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.605.586 [runtime.cc:2855] 619447 startAicpuExecutor: TsdOpen devId=7, device mode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.605.602 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.643 [client_manager.cpp:160][GetPlatformInfo][tid:619447] [TsdClient] mode[1] chipType[5] [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.605.662 [hdc_core.c:1982][ascend][curpid:619447,619579][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4242; pid=619447) [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.605.683 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619447] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:43.605.688 [hdc_api.cpp:671][tid:619579] connect succ, peer_node: 0, peer_devid: 7 [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.695 [client_manager.cpp:582][GetVisibleDevices][tid:619447] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.710 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.726 [client_manager.cpp:125][GetInstance][tid:619447] [ClientManager] GetInstance, deviceId[7], g_runningMode[0], begin saving instance [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.754 [client_manager.cpp:435][IsHostEnvironment][tid:619447] [TsdClient] logicDeviceId is [7], hostaicpunum[0] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.771 [client_manager.cpp:464][GetClientRunMode][tid:619447] runningMode:0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.781 [client_manager.cpp:127][GetInstance][tid:619447] [ClientManager] Current mode:1 [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.605.794 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619447] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.806 [client_manager.cpp:347][GetProfilingMode][tid:619447] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.605.840 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619447] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.856 [process_mode_base.cpp:149][GetLogLevel][tid:619447] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.926 [atrace_client_communication.c:177](tid:619579) send [start] msg. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.952 [process_mode_base.cpp:161][GetLogLevel][tid:619447] [TsdClient] Get Log Level[001] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.970 [process_mode_manager.cpp:71][OpenProcess][tid:619447] [ProcessModeManager] enter into open process deviceId[7] rankSize[0] [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.605.977 [atrace_client_communication.c:189](tid:619579) send [start] msg success. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.605.987 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619447] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.043 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[.], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.145 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[scene.info], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.233 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.331 [client_manager.cpp:264][CheckPackageExists][tid:619447] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.348 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[ascend_install.info], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.423 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[version.info], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.495 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[..], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.567 [client_manager.cpp:261][CheckPackageExists][tid:619447] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.737 [client_manager.cpp:268][CheckPackageExists][tid:619447] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.754 [client_manager.cpp:283][CheckPackageExists][tid:619447] [TsdClient][deviceId=7] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.767 [client_manager.cpp:286][CheckPackageExists][tid:619447] [TsdClient][deviceId=7] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.606.821 [devdrv_hccl.c:103][ascend][curpid:619447,619447][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619447) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.838 [process_mode_manager.cpp:396][InitTsdClient][tid:619447] [TsdClient][deviceId=7] get process sign success, procPid[619447] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.853 [process_mode_manager.cpp:403][InitTsdClient][tid:619447] [TsdClient] deviceId[7] begin to init hdc client [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.889 [hdc_common.cpp:67][InitMsgSize][tid:619447] HdcCommon::InitMsgSize Start [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.905 [hdc_common.cpp:80][InitMsgSize][tid:619447] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.920 [hdc_client.cpp:184][CreateHdcSession][tid:619447] HdcClient::CreateHdcSession Start [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.606.996 [hdc_client.cpp:208][CreateHdcSession][tid:619447] [HdcClient] deviceId: 7 connect session [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.607.026 [hdc_core.c:1982][ascend][curpid:619447,619447][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4243; pid=619447) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.043 [hdc_client.cpp:219][CreateHdcSession][tid:619447] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.057 [hdc_client.cpp:230][CreateHdcSession][tid:619447] [HdcClient] deviceId: 7 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.075 [version_verify.cpp:36][SetVersionInfo][tid:619447] VersionVerify: send client version to server [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.120 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.144 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.168 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.607.201 [internal_api.cpp:101][GetScheduleEnv][tid:619447] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.605 [version_verify.cpp:68][PeerVersionCheck][tid:619447] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.625 [version_verify.cpp:89][ParseVersionInfo][tid:619447] VersionVerify: pass client version info success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.642 [hdc_client.cpp:275][CheckHdcConnection][tid:619447] Service[2] create hdc success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.656 [version_verify.cpp:122][SpecialFeatureCheck][tid:619447] VersionVerify: new type[35], supported [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.607.672 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619447] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.704 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619447] [TsdClient][deviceId=7] [sessionId=1] wait package info respond [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.607.720 [hdc_client.cpp:154][TsdRecvData][tid:619447] recv timeout is:150000 ms [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.712 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619447] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.730 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619447] [PackageInfoMsgProc] [sessionID=1] deviceId[7], checkCode[56684994] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.747 [hdc_client.cpp:172][TsdRecvData][tid:619447] [deviceId=7] the Tsd recv data already [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.762 [version_verify.cpp:122][SpecialFeatureCheck][tid:619447] VersionVerify: new type[64], supported [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.777 [hdc_client.cpp:411][Destroy][tid:619447] enter HdcClient::Destroy() function [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.608.791 [hdc_client.c:417][ascend][curpid:619447,619447][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4243) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.816 [hdc_client.cpp:394][DestroyClient][tid:619447] begin drvHdcClientDestroy [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.832 [hdc_client.cpp:399][DestroyClient][tid:619447] end drvHdcClientDestroy [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.846 [hdc_client.cpp:377][ClearClientPtr][tid:619447] begin HdcClient::ClearClientPtr [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.860 [hdc_client.cpp:385][ClearClientPtr][tid:619447] end HdcClient::ClearClientPtr [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.871 [hdc_client.cpp:418][Destroy][tid:619447] end HdcClient::Destroy() function [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.892 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619447] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.909 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619447] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.608.929 [devdrv_hccl.c:103][ascend][curpid:619447,619447][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619447) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.944 [process_mode_manager.cpp:396][InitTsdClient][tid:619447] [TsdClient][deviceId=7] get process sign success, procPid[619447] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.956 [process_mode_manager.cpp:403][InitTsdClient][tid:619447] [TsdClient] deviceId[7] begin to init hdc client [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.972 [hdc_common.cpp:67][InitMsgSize][tid:619447] HdcCommon::InitMsgSize Start [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.608.995 [hdc_common.cpp:80][InitMsgSize][tid:619447] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.009 [hdc_client.cpp:184][CreateHdcSession][tid:619447] HdcClient::CreateHdcSession Start [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.063 [hdc_client.cpp:208][CreateHdcSession][tid:619447] [HdcClient] deviceId: 7 connect session [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.609.092 [hdc_core.c:1982][ascend][curpid:619447,619447][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4244; pid=619447) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.109 [hdc_client.cpp:219][CreateHdcSession][tid:619447] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.121 [hdc_client.cpp:230][CreateHdcSession][tid:619447] [HdcClient] deviceId: 7 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.135 [version_verify.cpp:36][SetVersionInfo][tid:619447] VersionVerify: send client version to server [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.151 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.165 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.179 [version_verify.cpp:52][SetVersionInfo][tid:619447] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.427 [version_verify.cpp:68][PeerVersionCheck][tid:619447] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.446 [version_verify.cpp:89][ParseVersionInfo][tid:619447] VersionVerify: pass client version info success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.460 [hdc_client.cpp:275][CheckHdcConnection][tid:619447] Service[2] create hdc success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.478 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619447] [TsdClient] tsd get process sign successfully, procpid[619447] signSize[48] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.494 [version_verify.cpp:114][SpecialFeatureCheck][tid:619447] VersionVerify: previous type[6], supported [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.523 [process_mode_manager.cpp:88][OpenProcess][tid:619447] [ProcessModeManager] deviceId[7] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619447,python3.9):2025-04-03-09:35:43.609.540 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619447] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.609.552 [hdc_client.cpp:154][TsdRecvData][tid:619447] recv timeout is:150000 ms [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.819.465 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619447] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.819.488 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619447] [TsdClient] DeviceMsgProc recvMsg realDeviceId[7] msgType[8] localDevId[7] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.819.501 [hdc_client.cpp:172][TsdRecvData][tid:619447] [deviceId=7] the Tsd recv data already [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.819.527 [devmm_svm.c:132][ascend][curpid:619447,619447][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=7) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.821.711 [devmm_virt_interface.c:450][ascend][curpid:619447,619447][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.822.029 [drv_log_user.c:623][ascend][curpid:619447,619447][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=7; devid=7; vfid=0; hostpid=619447; devpid=30963) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.822.066 [devmm_svm.c:122][ascend][curpid:619447,619447][drv][devmm][devmm_set_device_info]Device info. (devid=7; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.822.340 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619447] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.822.361 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619447] enter into OpenInHost deviceid[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.822.392 [client_manager.cpp:435][IsHostEnvironment][tid:619447] [TsdClient] logicDeviceId is [7], hostaicpunum[0] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.822.408 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619447] host cpu not support [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.822.419 [process_mode_manager.cpp:121][OpenProcess][tid:619447] [TsdClient][deviceId=7] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.822.434 [runtime.cc:2879] 619447 startAicpuExecutor: userDeviceId=7, devId=7, tsdOpen ret=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.822.448 [runtime.cc:2891] 619447 startAicpuExecutor: TsdOpen success, devId=7, ret=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.822.464 [runtime.cc:3627] 619447 DeviceRetain: new dev ok id=7, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.822.481 [raw_device.cc:936] 619447 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.822.496 [npu_driver.cc:2968] 619447 DeviceOpen: Open device start, device_id=7, tsId=0. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.822.581 [trs_sqcq.c:458][ascend][curpid:619447,619447][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=7) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.823.038 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=6; num=2044) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.823.505 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=7; num=2044) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.823.744 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=10; num=1024) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.823.996 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=11; num=1024) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.824.498 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=12; num=2048) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.824.527 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=8; num=16) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.824.550 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=9; num=16) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.824.571 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=13; num=4) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.824.588 [trs_sqcq.c:394][ascend][curpid:619447,619447][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=7; tsId=0; id_type=14; num=10) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.826.602 [trs_dev_drv.c:302][ascend][curpid:619447,619447][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=7; fd=36; mmapFd=37) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.826.642 [devmm_svm.c:132][ascend][curpid:619447,619447][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=7) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.826.664 [devmm_svm.c:122][ascend][curpid:619447,619447][drv][devmm][devmm_set_device_info]Device info. (devid=7; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.826.736 [npu_driver.cc:3011] 619447 DeviceOpen: Open device success, device_id=7, tsId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.826.756 [raw_device.cc:292] 619447 Init: chipType:0x5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.826.801 [raw_device.cc:304] 619447 Init: isAddrFlat:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.826.821 [raw_device.cc:320] 619447 Init: GetDevInfo, device_id=7, hardwareVersion=0x503 [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.826.838 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.826.855 [npu_driver.cc:1578] 619447 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.827.828 [raw_device.cc:352] 619447 Init: Get phy info, phy chip id 7, phy die id 0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.827.850 [runtime.cc:2771] 619447 GetTsdQos: [TSD] start get qos, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.827.864 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.879 [tsd_client.cpp:189][TsdCapabilityGet][tid:619447] TsdCapabilityGet Begin. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.892 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.907 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.919 [process_mode_manager.cpp:1042][CapabilityGet][tid:619447] [ProcessModeManager] enter into CapabilityGet process deviceId[7] type[0] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.933 [process_mode_manager.cpp:386][InitTsdClient][tid:619447] [TsdClient] tsd client has been inited before [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.949 [version_verify.cpp:114][SpecialFeatureCheck][tid:619447] VersionVerify: previous type[38], supported [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.827.985 [process_mode_manager.cpp:1065][CapabilityGet][tid:619447] [TsdClient][deviceId=7] [sessionId=1] [type=0]send capability success [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.828.003 [hdc_client.cpp:154][TsdRecvData][tid:619447] recv timeout is:150000 ms [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.828.957 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619447] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.828.977 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619447] [TsdClient] PidQosMsgProc recvMsg realDeviceId[7] msgType[39] localDevId[7] rspCode[0],pidqos[1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.828.994 [hdc_client.cpp:172][TsdRecvData][tid:619447] [deviceId=7] the Tsd recv data already [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.829.008 [process_mode_manager.cpp:1070][CapabilityGet][tid:619447] [TsdClient][logicDeviceId_=7][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.025 [runtime.cc:2782] 619447 GetTsdQos: [TSD] get qos success, userDeviceId=7, devId=7, qos=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.057 [engine.cc:78] 619447 Engine: Constructor. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.072 [stars_engine.cc:51] 619447 StarsEngine: Constructor. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.085 [engine_factory.cc:26] 619447 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.099 [engine_factory.cc:38] 619447 CreateEngine: Engine init success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.112 [raw_device.cc:364] 619447 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.127 [spm_pool.cc:42] 619447 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.256 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.274 [spm_pool.cc:55] 619447 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.288 [raw_device.cc:380] 619447 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.829.304 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.321 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.829.334 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.829.347 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.836.764 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.836.795 [task_allocator.cc:46] 619447 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.836.831 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.836.847 [h2d_copy_mgr.cc:45] 619447 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.836.861 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.836.878 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.837.972 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.837.995 [task_allocator.cc:46] 619447 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.009 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.022 [h2d_copy_mgr.cc:45] 619447 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.838.038 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.055 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.846 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.867 [task_allocator.cc:46] 619447 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.908 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.924 [h2d_copy_mgr.cc:45] 619447 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.940 [uma_arg_loader.cc:86] 619447 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.956 [task_allocator.cc:46] 619447 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.982 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.838.997 [uma_arg_loader.cc:93] 619447 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.839.823 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.839.842 [uma_arg_loader.cc:102] 619447 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.839.856 [uma_arg_loader.cc:104] 619447 Init: ALLOC PCIE is support[1] [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.839.870 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.839.887 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.011 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.029 [h2d_copy_mgr.cc:45] 619447 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.044 [raw_device.cc:393] 619447 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.060 [raw_device.cc:398] 619447 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.076 [task_allocator.cc:46] 619447 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.275 [buffer_allocator.cc:61] 619447 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.486 [raw_device.cc:109] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.867 [raw_device.cc:121] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.886 [raw_device.cc:425] 619447 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.840.939 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.021 [stream.cc:2246] 619447 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.058 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.076 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.547 [npu_driver.cc:6125] 619447 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.622 [npu_driver.cc:6137] 619447 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=339, streamId=0, logicCq=339, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.649 [npu_driver.cc:6189] 619447 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=0, logicCq=339. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.677 [npu_driver.cc:5916] 619447 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=fb3eb000, addr=dffffb3eb000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.692 [stream.cc:653] 619447 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.745 [stream.cc:276] 619447 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.761 [stream.cc:688] 619447 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=7, streamResId=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.779 [stream_state_callback_manager.cc:45] 619447 Notify: not reg stream state callback func! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.793 [raw_device.cc:593] 619447 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.817 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.839 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=16 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.854 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.947 [tsch_stream.cc:45] 619447 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=7 streamResId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.841.964 [raw_device.cc:1219] 619447 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.043 [osal.cc:309] 619447 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.167 [osal.cc:182] 619447 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.210 [osal.cc:309] 619447 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.257 [stars_engine.cc:1785] 619596 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.286 [osal.cc:182] 619447 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.303 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.303 [stars_engine.cc:201] 619597 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.842.324 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.842.343 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.842.358 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.384 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.437 [npu_driver.cc:6306] 619596 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.458 [stars_engine.cc:1855] 619596 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.474 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.632 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.667 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.700 [task_info.cc:1596] 619447 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.717 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.731 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.759 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.774 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.789 [stars_engine.cc:1122] 619447 SyncTask: Task Wait: device_id=7, stream_id=0, task_id=0, logicCqId=339. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.830 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.845 [stars_engine.cc:1071] 619447 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.871 [stars_engine.cc:1017] 619447 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.889 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.912 [stars_engine.cc:1027] 619447 ProcReport: taskType=97. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.933 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.946 [stars_engine.cc:1570] 619447 ProcLogicCqReport: CQE warning, device_id=7, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.962 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.977 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.842.989 [task_info.cc:1614] 619447 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.000 [raw_device.cc:1305] 619447 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.031 [stars_engine.cc:1351] 619447 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.049 [npu_driver.cc:5618] 619447 GetSqEnable: deviceId=7, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.063 [stars_engine.cc:1365] 619447 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.076 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.089 [npu_driver.cc:5595] 619447 GetSqTail: deviceId=7, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.100 [npu_driver.cc:5406] 619447 SetSqHead: deviceId=7, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.116 [npu_driver.cc:5357] 619447 EnableSq: deviceId=7, tsId=0, sqId=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.131 [stars_engine.cc:1430] 619447 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.154 [npu_driver.cc:5618] 619447 GetSqEnable: deviceId=7, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.168 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.180 [raw_device.cc:1331] 619447 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.843.192 [raw_device.cc:610] 619447 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.584 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.688 [ringbuffer_maintain_task.cc:98] 619447 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x1580 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.712 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.732 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.762 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.779 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.798 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.819 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.881 [stars_engine.cc:622] 619447 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=339, retCode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.899 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.912 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.926 [stream.cc:2886] 619447 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.940 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.957 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.971 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.844.988 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.013 [stars_engine.cc:437] 619447 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.030 [stream.cc:2886] 619447 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.103 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.125 [task_info.cc:111] 619447 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.143 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.173 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.189 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.203 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.218 [stars_engine.cc:1122] 619447 SyncTask: Task Wait: device_id=7, stream_id=0, task_id=2, logicCqId=339. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.372 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.412 [stars_engine.cc:1071] 619447 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.428 [stars_engine.cc:1017] 619447 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.446 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.461 [stars_engine.cc:1027] 619447 ProcReport: taskType=29. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.474 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.492 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.507 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.532 [npu_driver.cc:5618] 619447 GetSqEnable: deviceId=7, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.547 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.845.561 [raw_device.cc:1232] 619447 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.845.584 [atrace_api.c:40](tid:619447) AtraceCreate start. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.712 [trace_rb_log.c:45](tid:619447) init ring buffer time finished, realTime 1743644143845647782, monotonicTime 4211305558988, freq 99996 kHz, minutesWest 480 minutes [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.730 [trace_rb_log.c:137](tid:619447) [RUNTIME_ATRACE_DEV7_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.747 [tracer_mgr_operate.c:66](tid:619447) create object RUNTIME_ATRACE_DEV7_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.760 [atrace_api.c:44](tid:619447) AtraceCreate end. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.803 [trace_event.c:70](tid:619447) limitedNum 1, processType 0 [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.846.818 [trace_event.c:372](tid:619447) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.846.835 [runtime.cc:3159] 619447 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619447,python3.9):2025-04-03-09:35:43.846.849 [client_manager.cpp:196][SetProfilingCallback][tid:619447] [TsdClient] set profiling callback success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.846.891 [runtime.cc:3364] 619447 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.846.940 [context.cc:109] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.034 [context.cc:121] 619447 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.048 [context.cc:276] 619447 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.106 [raw_device.cc:803] 619447 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.120 [npu_driver_dcache_lock.cpp:39] 619447 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.249 [raw_device.cc:528] 619447 AllocStackPhyBase: AllocStackPhyBase device_id=7, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.298 [raw_device.cc:232] 619447 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.319 [runtime.cc:3393] 619447 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.335 [runtime.cc:3560] 619447 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.355 [api_impl.cc:2987] 619447 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.368 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.381 [api_impl.cc:2992] 619447 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:43.847.399 [awatchdog_core.c:175](tid:619447) create watchdog for id : 65543, tid : 619447, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.847.412 [api_impl.cc:3025] 619447 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.847.444 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.847.459 [acl.cpp:121]619447 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.849.985 [api_impl.cc:3476] 619447 GetDeviceInfo: get device info, deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.019 [api_impl.cc:3476] 619447 GetDeviceInfo: get device info, deviceId=7 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.850.059 [device.cpp:133]619447 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 7, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.850.078 [device.cpp:154]619447 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.850.094 [context.cpp:87]619447 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.850.116 [context.cpp:98]619447 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.850.134 [stream.cpp:48]619447 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.167 [api_impl.cc:1120] 619447 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.222 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.254 [stream.cc:2246] 619447 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.289 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.317 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.615 [npu_driver.cc:6125] 619447 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.668 [npu_driver.cc:6137] 619447 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=340, streamId=2, logicCq=340, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.688 [npu_driver.cc:6189] 619447 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=2, logicCq=340. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.703 [npu_driver.cc:5916] 619447 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f40c9000, addr=dffff40c9000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.716 [stream.cc:653] 619447 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.850.763 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.781 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.850.945 [stream.cc:291] 619447 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.384 [stream.cc:688] 619447 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=7, streamResId=65535 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.422 [stream_state_callback_manager.cc:45] 619447 Notify: not reg stream state callback func! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.441 [api_impl.cc:1162] 619447 StreamCreate: Succ, flags=1536, stream_id=2, context=aaab0ca695c0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.853.468 [stream.cpp:189]619447 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.495 [stream.cc:3161] 619447 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.853.511 [stream.cpp:195]619447 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.853.574 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab0d9d25f0. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.853.616 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.853.633 [stream.cpp:48]619447 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.655 [api_impl.cc:1120] 619447 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.692 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.732 [stream.cc:2246] 619447 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.776 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.853.793 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.050 [npu_driver.cc:6125] 619447 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.104 [npu_driver.cc:6137] 619447 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=341, streamId=3, logicCq=341, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.144 [npu_driver.cc:6189] 619447 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=3, logicCq=341. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.165 [npu_driver.cc:5916] 619447 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f40a8000, addr=dffff40a8000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.182 [stream.cc:653] 619447 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619447,python3.9):2025-04-03-09:35:43.854.243 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.261 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.854.572 [stream.cc:291] 619447 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.212 [stream.cc:688] 619447 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=7, streamResId=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.232 [stream_state_callback_manager.cc:45] 619447 Notify: not reg stream state callback func! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.245 [api_impl.cc:1162] 619447 StreamCreate: Succ, flags=1536, stream_id=3, context=aaab0ca695c0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.261 [stream.cpp:189]619447 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.282 [stream.cc:3161] 619447 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.297 [stream.cpp:195]619447 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.855.335 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaab0db033f0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.452 [callback.cpp:25]619447 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471797227424. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.479 [npu_driver.cc:462] 619447 EventIdAlloc: event id alloc begin, device_id=7, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.491 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.526 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.542 [npu_driver.cc:483] 619447 EventIdAlloc: Event id alloc success, device_id=7, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.544 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.557 [event.cc:347] 619447 GenEventId: Event id alloc success, device_id=7, tsId=0, eventFlag=1, event_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.576 [subscribe.cc:54] 619447 AssignGroupId: give groupId=320 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.589 [npu_driver.cc:3775] 619447 SqCqAllocate: deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.612 [npu_driver.cc:3785] 619447 SqCqAllocate: alloc success, deviceId=7 sqId=118, cqId=118. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.621 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.634 [subscribe.cc:131] 619447 Insert: callback subscribe: groupId=320, threadId=281471797227424, devId=7, tsId=0, streamId=2, sqId=118, cqId=118 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.662 [callback.cpp:31]619447 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471797227424. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.661 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.689 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.855.692 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab0d9d25f0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.765 [callback.cpp:25]619447 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471788773280. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.787 [npu_driver.cc:462] 619447 EventIdAlloc: event id alloc begin, device_id=7, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.796 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.807 [npu_driver.cc:483] 619447 EventIdAlloc: Event id alloc success, device_id=7, ts_id=0, event_id=1, eventFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.836 [event.cc:347] 619447 GenEventId: Event id alloc success, device_id=7, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.838 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.850 [subscribe.cc:54] 619447 AssignGroupId: give groupId=321 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.856 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.871 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.863 [npu_driver.cc:3775] 619447 SqCqAllocate: deviceId=7, tsId=0, groupId=321. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.886 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.896 [npu_driver.cc:3785] 619447 SqCqAllocate: alloc success, deviceId=7 sqId=119, cqId=119. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.911 [subscribe.cc:131] 619447 Insert: callback subscribe: groupId=321, threadId=281471788773280, devId=7, tsId=0, streamId=3, sqId=119, cqId=119 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.855.932 [callback.cpp:31]619447 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471788773280. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.855.938 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.855.958 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaab0db033f0. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.855.981 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.070 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.856.399 [npu_driver.cc:2616] 619447 MemGetInfoEx: memory info type=1, type=2, free=8863031296, hug_free=56245616640 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.461 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.559 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.609 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.639 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.662 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62092M, Reserved MOC size for Other Components(HCCL/rts/etc.):3892M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58200M. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.856.687 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58200M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.856.719 [event.cpp:260]619447 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.856.740 [api_impl.cc:5713] 619447 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.821 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.846 [timeout_set_task.cc:115] 619447 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.862 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.879 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.905 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.918 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.935 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.953 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.857.997 [stars_engine.cc:622] 619447 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=339, retCode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.016 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.033 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.046 [stream.cc:2886] 619447 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.059 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.086 [stream.cc:2886] 619447 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.101 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.113 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.127 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.141 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.165 [stars_engine.cc:437] 619447 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.182 [stream.cc:2886] 619447 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.197 [event.cpp:262]619447 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.215 [event.cpp:269]619447 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.234 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.252 [timeout_set_task.cc:115] 619447 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.268 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.283 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.302 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.318 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.333 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.347 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.359 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.373 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.387 [stream.cc:2886] 619447 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.400 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.414 [stream.cc:2886] 619447 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.428 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.442 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.455 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.467 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.493 [stars_engine.cc:437] 619447 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.508 [stream.cc:2886] 619447 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.522 [event.cpp:277]619447 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.548 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.586 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.607 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 7 has been set [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.625 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.643 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.694 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.725 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619447,ffffb243e020,python3.9):2025-04-03-09:35:43.858.778 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.911 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.858.936 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.972 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.858.996 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:43.859.012 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.859.029 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.859.187 [acl.cpp:288]619447 aclInit: start to execute aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.859.231 [acl.cpp:298]619447 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.859.366 [acl.cpp:242]619447 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.868.449 [error_manager.cc:673]619447 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:43.869.208 [acl.cpp:315]619447 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:43.869.290 [prof_atls_plugin.cpp:210] (tid:619447) Module[45] register callback of ctrl handle. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.869.335 [ge_executor.cc:273]619447 Initialize:Init GeExecutor begin. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.869.411 [op_lib_register.cc:117]619447 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.509 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.547 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.580 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.613 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.644 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.674 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.704 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.733 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.763 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.792 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.822 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.862 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.892 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.922 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.952 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.869.982 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.870.011 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.870.041 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.870.070 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.870.099 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.139 [op_lib_register.cc:165]619447 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.179 [plugin_manager.cc:616]619447 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.203 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.236 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.259 [plugin_manager.cc:624]619447 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619447,python3.9):2025-04-03-09:35:43.870.276 [plugin_manager.cc:206]619447 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.162 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.206 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.223 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.236 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.283 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.636 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.670 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.687 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.699 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.735 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.776 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.793 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.871.832 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.848 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.871.867 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.230 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.270 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.293 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.306 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.341 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.687 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.721 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.738 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.751 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.787 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.825 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.841 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.872.855 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.869 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.872.886 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.236 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.269 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.286 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.299 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.334 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.679 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.711 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.727 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.742 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.779 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.816 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.832 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.873.844 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.859 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.873.875 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.216 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.248 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.264 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.276 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.324 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.663 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.696 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.714 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.729 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.764 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.800 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.816 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.874.827 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.841 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.874.855 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.197 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.229 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.245 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.259 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.295 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.636 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.668 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.710 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.724 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.759 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.794 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.810 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.875.822 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.836 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.853 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.869 [plugin_manager.cc:218]619447 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.890 [plugin_manager.cc:422]619447 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.905 [plugin_manager.cc:155]619447 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.927 [plugin_manager.cc:158]619447 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619447,python3.9):2025-04-03-09:35:43.875.943 [plugin_manager.cc:426]619447 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.286 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.876.324 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.876.356 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.876.371 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.411 [plugin_manager.cc:439]619447 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.432 [plugin_manager.cc:175]619447 ReversePathString:Enter ReversePathString schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.455 [plugin_manager.cc:187]619447 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.782 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.813 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:43.876.835 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619447,python3.9):2025-04-03-09:35:43.876.880 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619447,python3.9):2025-04-03-09:35:43.891.595 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.210 [op_tiling_registry.cc:162]619447 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.329 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.427 [op_tiling_registry.cc:162]619447 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.530 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.639 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.735 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.851 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.870 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.897 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619447,python3.9):2025-04-03-09:35:43.892.992 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.084 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.173 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.267 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.364 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.692 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.780 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.890 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.893.993 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.088 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.187 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.289 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.472 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.575 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.678 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.777 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.796 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.894.896 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.017 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.035 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.138 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.238 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.335 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.530 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.700 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.794 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.894 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619447,python3.9):2025-04-03-09:35:43.895.996 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619447,python3.9):2025-04-03-09:35:43.896.431 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.896.537 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.896.644 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619447,python3.9):2025-04-03-09:35:43.896.819 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619447,python3.9):2025-04-03-09:35:43.896.922 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.034 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.134 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.232 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.327 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.430 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.539 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.634 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.651 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.752 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.865 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.960 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619447,python3.9):2025-04-03-09:35:43.897.978 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.072 [op_tiling_registry.cc:122]619447 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.165 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.267 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.364 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.469 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.663 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.835 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.911 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.898.984 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.063 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.136 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.163 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.247 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.300 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.450 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.685 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.761 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.842 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.870 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.943 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.969 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.899.992 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.065 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.147 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.233 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.314 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.391 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.465 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.540 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.612 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.688 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.758 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.859 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.900.936 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.901.010 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.901.084 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.901.162 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.901.237 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.901.320 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.905.282 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619447,python3.9):2025-04-03-09:35:43.905.312 [op_tiling_registry.cc:148]619447 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619447,python3.9):2025-04-03-09:35:43.906.641 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.906.760 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.906.879 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.140 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.263 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.369 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.620 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.907.877 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.022 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.134 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.269 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.398 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.512 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.625 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.802 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.908.909 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.045 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.176 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.314 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.422 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.532 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.681 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.796 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.909.902 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.017 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.140 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.253 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.370 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.590 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.704 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.910.945 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.058 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.199 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.399 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.514 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.709 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.857 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.911.979 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.121 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.151 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.268 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.377 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.404 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.513 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.622 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.912.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.037 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.066 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.466 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.572 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.685 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.800 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.913.911 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.094 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.243 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.394 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.559 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.680 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.789 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.914.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.019 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.131 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.380 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.547 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.604 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff1d345d70, registered count:1 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.766 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.789 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff1d34e248, registered count:2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.915.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.026 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.180 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.293 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.410 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.523 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.743 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.857 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.964 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.916.991 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.113 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.225 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.446 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.551 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.666 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.770 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.917.882 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.026 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.054 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.081 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.250 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.367 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.587 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.693 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.851 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.918.968 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.079 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.372 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.609 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.720 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.836 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.919.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.206 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.314 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.348 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.617 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.841 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.920.956 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.063 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.288 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.397 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.586 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.697 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.921.819 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.360 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.407 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.452 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.583 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.702 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.817 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.924.929 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.035 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.142 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.254 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.361 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.511 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.657 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.777 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.806 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.925.910 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.019 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.128 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.235 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.445 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.713 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.855 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.926.970 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.927.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.927.218 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.927.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.928.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.928.956 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.929.394 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.929.966 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.930.083 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.930.196 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.930.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.930.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.930.758 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.932.893 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.933.695 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.933.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.933.997 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.934.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.935.090 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.935.241 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.935.698 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.935.810 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.935.923 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.037 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.173 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.291 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.402 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.512 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.626 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.735 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.852 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.936.995 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.224 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.400 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.516 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.628 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.743 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.937.897 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.068 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.185 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.297 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.326 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.614 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.729 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.938.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.031 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.179 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.293 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.404 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.516 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.544 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.658 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.687 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.794 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.903 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.939.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.045 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.170 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.576 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.940.913 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.254 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.435 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.616 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.740 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.861 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.941.971 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.394 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.506 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.623 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.840 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.942.976 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.943.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.943.301 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.944.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.946.914 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.947.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.947.301 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.947.600 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.947.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.947.916 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.948.712 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.948.871 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.005 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.157 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.306 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.573 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.720 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.855 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.949.977 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.089 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.178 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.473 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.531 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.556 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.581 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.609 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.635 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.661 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.686 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.711 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.738 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.851 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.876 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.901 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.950 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.950.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.005 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.053 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.129 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.153 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.199 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.224 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.249 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.274 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.300 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.327 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.353 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.387 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.412 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.437 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.464 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.491 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.516 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.566 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.617 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.668 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.692 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.718 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.742 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.766 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.792 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.819 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.843 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.866 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.951.888 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.191 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.368 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.398 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.424 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.450 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.475 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.615 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.719 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.952.928 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.141 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.294 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.403 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.552 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.667 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.778 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.807 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.837 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.944 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.973 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.953.999 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.103 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.252 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.420 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.564 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.674 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.850 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.954.964 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.139 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.248 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.354 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.385 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.410 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.562 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.723 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.955.867 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.009 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.195 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.450 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.600 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.754 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.956.929 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.160 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.273 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.507 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.653 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.957.910 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.958.017 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.958.134 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.958.247 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.958.354 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.958.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.219 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.445 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.475 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.591 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.700 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.806 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.959.920 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.143 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.276 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.537 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.653 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.765 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.960.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.058 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.171 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.278 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.460 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.633 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.746 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.854 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.961.969 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.082 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.191 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.298 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.404 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.550 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.663 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.962.812 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.222 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.384 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.556 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.777 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.963.977 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.232 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.281 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.313 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.338 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.364 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.964.388 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.965.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.965.475 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.965.883 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.966.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.966.277 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.966.659 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.966.785 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.966.953 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.969.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.971.328 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.971.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.971.571 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.971.685 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.971.964 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.119 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.315 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.633 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.667 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.696 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.722 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.747 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.922 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.972.952 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.109 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.586 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.795 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.973.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.104 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.268 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.438 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.757 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.786 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.974.929 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.236 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.269 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.390 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.419 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.538 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.567 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.680 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.792 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.975.908 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.027 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.135 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.253 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.434 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.636 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.976.986 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.977.344 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.977.546 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.977.867 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.070 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.247 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.349 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.384 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.487 [op_tiling_registry.cc:134]619447 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.518 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.659 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.689 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.719 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.978.746 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.979.768 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.202 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.355 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.687 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.828 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.980.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.116 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.147 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.443 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.590 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.762 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.981.909 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.982.280 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.982.619 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.982.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.983.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.984.050 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.984.460 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.984.985 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.143 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.291 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.561 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.709 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.741 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.985.878 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.179 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.289 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.436 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.587 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.702 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.986.998 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.190 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.448 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.558 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.711 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.987.881 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.245 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.278 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.431 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.988.612 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.990.104 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.995.718 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.995.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.998.568 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.998.816 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.999.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.999.404 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.999.602 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.999.796 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:43.999.956 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.115 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.315 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.545 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.788 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.000.911 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.061 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.517 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.825 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.859 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.001.968 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.091 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.262 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.446 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.627 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.777 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.002.893 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.003.007 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.003.036 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.003.793 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.003.988 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.143 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.439 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.468 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.590 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.702 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.760 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.004.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.090 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.409 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.591 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.622 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.650 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.829 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.005.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.684 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.786 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.812 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.885 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.971 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.008.995 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.070 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.233 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.269 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.379 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.400 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.421 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.444 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.626 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.009.647 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.010.831 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff1dc98a10, registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.010.868 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff1dc98c70, registered count:1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.011.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.011.792 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.013.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.013.698 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.013.779 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.014.174 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.014.633 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.048 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.215 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.417 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.653 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.046.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.047.441 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.047.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.047.800 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.047.888 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.048.047 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.048.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.048.497 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.048.927 [op_tiling_manager.cc:79]619447 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.048.991 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.217 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.263 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.329 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.384 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.424 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.470 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.049.527 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.544 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.580 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.604 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.625 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.650 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.050.984 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff501509b0, registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.008 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.029 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.045 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.064 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.374 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff50153da0, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.398 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.418 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.435 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.454 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.051.744 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.051.810 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.051.902 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.053.743 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff5010a2c0, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.053.775 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.053.793 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.053.813 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.053.832 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.153 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff5010e978, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.178 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.197 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.215 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.233 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.498 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.546 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.643 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.706 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.745 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.788 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.821 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.860 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.899 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.054.933 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.054.971 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.009 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.045 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.081 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.126 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.158 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.196 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.233 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.263 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.300 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.334 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.371 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.407 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.445 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.482 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.519 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.557 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.588 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.624 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.661 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.697 [op_tiling_manager.cc:73]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.734 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.774 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.807 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.844 [op_tiling_manager.cc:58]619447 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.055.889 [op_tiling_manager.cc:67]619447 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.923 [op_tiling_manager.cc:75]619447 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.055.941 [op_tiling_manager.cc:109]619447 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [185763] micro seconds. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.056.029 [plugin_manager.cc:639]619447 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.056.047 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.056.072 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.056.127 [host_cpu_engine.cc:254]619447 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.060.859 [host_cpu_engine.cc:273]619447 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619447,python3.9):2025-04-03-09:35:44.060.932 [host_cpu_engine.cc:254]619447 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.067.158 [host_cpu_engine.cc:267]619447 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619447,python3.9):2025-04-03-09:35:44.068.002 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619447] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.036 [folding.cc:404][InitCpuConstantFoldingNew][tid:619447]:"Init cpu constant folding begin." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.077 [folding.cc:408][InitCpuConstantFoldingNew][tid:619447]:"Number of registered ops: 234" [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.096 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Acos]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Acosh]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.131 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.147 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.176 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.196 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.208 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.222 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Add]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.238 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Addcdiv]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.267 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Addcmul]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.278 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[AffineGrid]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.291 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ApplyAdamV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.300 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Asin]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.314 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Asinh]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.328 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Assert]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.342 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Atan]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Atan2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.368 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Atanh]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.380 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[AvgPoolV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.394 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Betainc]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.407 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Bucketize]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.421 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CTCLossV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.435 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.446 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CacheSwapTable]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.460 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.473 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CaseCondition]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.487 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cast]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.501 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Ceil]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.514 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CheckNumerics]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.526 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ClipByValue]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.539 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ClipByValueV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.553 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Combinations]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.567 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CompareAndBitpack]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.579 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.592 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ConcatV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.607 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Conj]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.619 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ConjugateTranspose]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.633 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.657 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cosh]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.670 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[CropAndResize]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.685 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cross]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.699 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cummax]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.716 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cummin]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.730 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Cumsum]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.746 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Div]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.760 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[DynamicStitch]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.776 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EditDistance]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.789 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Elu]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.805 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EluGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.818 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.834 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.848 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.877 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.893 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.908 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.921 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.933 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.946 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.959 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.971 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.984 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.068.998 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.011 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingLookup]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.025 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.039 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.059 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.074 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.088 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.104 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Equal]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.117 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Expand]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.132 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ExpandDims]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.147 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Exponential]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.162 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.176 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Fill]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.191 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[FillDiagonal]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.204 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Fills]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.220 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Floor]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.233 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[FloorDiv]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.248 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[FloorMod]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.265 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GEMM]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.282 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GatherElements]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.295 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GatherV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.310 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Geometric]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.325 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GetDynamicDims]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.340 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Greater]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.355 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GreaterEqual]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.370 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GridSampler2D]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GridSampler3D]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.410 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.422 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Histogram]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.434 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.447 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Identity]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.468 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[IndexToAddr]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.480 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.493 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InitPartitionMap]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.508 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.520 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.535 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InvGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.550 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[InvertPermutation]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.562 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[IsClose]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.576 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[IsInf]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.590 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[IsNan]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.602 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Less]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.616 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[LinSpace]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.631 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[LinearSumAssignment]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.648 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Log]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Log1p]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.675 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.690 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.703 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.719 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MaskedFill]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.732 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MaskedScatter]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.747 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MaskedSelect]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.761 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.777 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MatMul]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.792 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.807 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MatrixDiagV3]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.821 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MinAreaPolygons]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.837 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Mul]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.851 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.892 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.907 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NLLLoss]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.922 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NLLLossGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.936 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NMSWithMask]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.949 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Neg]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.964 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.978 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NonZero]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.069.992 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.006 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.018 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[NotEqual]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.030 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[OnesLike]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.043 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[PSAMask]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.055 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[PSAMaskGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.069 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Pad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.084 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[PadD]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.099 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[PadV3]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.113 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[PadV3Grad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.129 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Pinverse]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.144 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Poisson]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.158 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Pow]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.171 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.187 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RNNTLoss]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.202 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RandomStandardNormal]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.216 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RandomUniform]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RealDiv]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.246 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ReduceAll]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.261 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ReduceMean]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.275 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ReduceSum]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.297 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Reshape]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.314 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Resize]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.329 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ResizeBilinear]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.344 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.356 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.370 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ResizeGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ResizeTrilinear]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.397 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ReverseSequence]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.410 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RightShift]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.423 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.439 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[RotatedNMS]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.453 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Round]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.467 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Rsqrt]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[STFT]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.493 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ScatterElements]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.519 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ScatterNdAdd]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ScatterNdMax]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.547 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ScatterNdMin]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.562 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SearchSorted]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.576 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SelectV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Sigmoid]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.599 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SigmoidGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.612 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Sign]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.627 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SilentCheck]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.639 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SilentCheckV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.653 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SilentCheckV3]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.666 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SliceWrite]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.682 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SoftmaxV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.701 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Sort]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.716 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.730 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SparseSegmentMean]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.744 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.760 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SparseSegmentSum]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.774 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SparseToDense]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.786 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SpatialTransformer]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.799 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SplitD]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.814 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SplitV]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.827 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Sqrt]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.842 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SqrtGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.855 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Square]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.868 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[SquaredDifference]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.885 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[StatelessBernoulli]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.899 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.912 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[StatelessRandperm]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.927 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[StridedSlice]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.940 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[StridedSliceV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.955 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Sub]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.971 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TableToResource]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.984 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Tan]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.070.998 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TanhGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.015 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TensorEqual]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.029 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.044 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ThreeNN]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.061 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Tile]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.074 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TileWithAxis]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.086 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TopK]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.105 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TopKPQDistance]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.120 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.134 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TopKV2D]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.148 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Trace]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.164 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TransData]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.176 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[TransDataRNN]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.189 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Tril]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.202 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Triu]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.215 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.243 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UninitPartitionMap]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.259 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UniqueConsecutive]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.272 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.288 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.301 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.318 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.330 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.345 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.358 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ViewCopy]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.371 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Voxelization]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.384 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[WarpAffine]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.401 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[ZerosLike]." [INFO] AICPU(619447,python3.9):2025-04-03-09:35:44.071.414 [folding.cc:413][InitCpuConstantFoldingNew][tid:619447]:"Register op[Zeta]." [INFO] OP(619447,python3.9):2025-04-03-09:35:44.071.434 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619447] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619447,python3.9):2025-04-03-09:35:44.071.451 [host_cpu_engine.cc:273]619447 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619447,python3.9):2025-04-03-09:35:44.071.580 [plugin_manager.cc:872]619447 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.075.043 [plugin_manager.cc:883]619447 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.116 [ops_kernel_executor_manager.cc:37]619447 Initialize:No library to load [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.134 [plugin_manager.cc:540]619447 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.147 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.181 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.203 [plugin_manager.cc:547]619447 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.075.220 [plugin_manager.cc:206]619447 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.113 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.168 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.189 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.206 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.257 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.667 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.702 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.718 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.731 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.768 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.810 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.835 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.850 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.868 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.900 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.915 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.076.927 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.076.940 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.077.336 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.370 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.386 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.399 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.077.435 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.077.832 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.865 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.882 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.896 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.077.932 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.969 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.077.996 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.012 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.078.028 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.058 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.073 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.087 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.078.103 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.078.491 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.523 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.539 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.078.553 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.078.593 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.078.982 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.014 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.031 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.043 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.079.075 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.112 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.138 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.152 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.079.165 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.195 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.211 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.225 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.079.242 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.079.629 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.661 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.677 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.079.689 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.079.724 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.114 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.146 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.170 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.183 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.219 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.267 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.283 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.295 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.311 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.340 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.354 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.365 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.379 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.773 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.806 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.823 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.080.837 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.080.874 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.277 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.310 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.326 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.339 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.376 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.420 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.436 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.451 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.466 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.497 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.512 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.081.523 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.538 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.552 [plugin_manager.cc:218]619447 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.575 [plugin_manager.cc:422]619447 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.589 [plugin_manager.cc:155]619447 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.607 [plugin_manager.cc:158]619447 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.081.620 [plugin_manager.cc:426]619447 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.011 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.082.052 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.082.067 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.082.082 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.117 [plugin_manager.cc:439]619447 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.137 [ge_executor.cc:259]619447 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.553 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.583 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.606 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.713 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.731 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.810 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.826 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.082.856 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.913 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.082.929 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.082.958 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.006 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.021 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.085 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.100 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.165 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.180 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.209 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.253 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.270 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.299 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.342 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.358 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.424 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.440 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.514 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.528 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.558 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.604 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.621 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.649 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.690 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.705 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.771 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.786 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.851 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.866 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.896 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.940 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.083.956 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.083.984 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.026 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.051 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.116 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.131 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.203 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.222 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.084.251 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.295 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.310 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.084.338 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.381 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.396 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.084.425 [opsproto_manager.cc:123]619447 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.466 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.530 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.546 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.559 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.573 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.084.587 [opsproto_manager.cc:147]619447 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.950 [operator_factory_impl.cc:215]619447 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619447,python3.9):2025-04-03-09:35:44.084.984 [operator_factory_impl.cc:269]619447 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619447,python3.9):2025-04-03-09:35:44.085.015 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.091.073 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619447,python3.9):2025-04-03-09:35:44.091.853 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.091.959 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.006 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.040 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.072 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.103 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.133 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.177 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.092.211 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.114.002 [operator_factory_impl.cc:368]619447 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619447,python3.9):2025-04-03-09:35:44.114.243 [operator_factory_impl.cc:284]619447 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.456 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.480 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.495 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.514 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.545 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619447,python3.9):2025-04-03-09:35:44.114.569 [operator_factory_impl.cc:254]619447 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619447,python3.9):2025-04-03-09:35:44.114.595 [operator_factory_impl.cc:340]619447 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619447,python3.9):2025-04-03-09:35:44.114.622 [operator_factory_impl.cc:307]619447 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.652 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.674 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.693 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.114.831 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.745 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.774 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.794 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.812 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.829 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.843 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.867 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.902 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.922 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.936 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.952 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.973 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.115.988 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.017 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.046 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.070 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.139 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.194 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.231 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.249 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.276 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.116.299 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.578 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.607 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.627 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.647 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.664 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.680 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.698 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.735 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.751 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.770 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.869 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.886 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.904 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.919 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.933 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.117.947 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.294 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.315 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.328 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.343 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.356 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.367 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.402 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.425 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.438 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.452 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.465 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.486 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.501 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.734 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.755 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.817 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.834 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.848 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.861 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.118.876 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.843 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.872 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.891 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.912 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.929 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.943 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.119.958 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.092 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.191 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.595 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.617 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.645 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.658 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.671 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.120.685 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.167 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.501 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.525 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.542 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.559 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.746 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.121.896 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.205 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.230 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.267 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.284 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.908 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.949 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.970 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.122.991 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.015 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.033 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.053 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.077 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.129 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.171 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.612 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.123.812 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.124.121 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.124.149 [operator_factory_impl.cc:222]619447 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.807 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.847 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.874 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.899 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.926 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.953 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.124.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.031 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.129 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.151 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.222 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.242 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.263 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.285 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.302 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.336 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.355 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.373 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.398 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.422 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.444 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.468 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.498 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.529 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.553 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.573 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.597 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.623 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.645 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.668 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.692 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.716 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.740 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.765 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.789 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.814 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.839 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.868 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.894 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.918 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.951 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.125.977 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.001 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.024 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.047 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.065 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.112 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.163 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.214 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.234 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.256 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.306 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.356 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.381 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.405 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.429 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.500 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.561 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.613 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.638 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.661 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.687 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.714 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.740 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.787 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.808 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.834 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.854 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.877 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.905 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.931 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.954 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.126.978 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.006 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.082 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.129 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.183 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.207 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.232 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.284 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.307 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.331 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.356 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.380 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.401 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.424 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.449 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.474 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.498 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.545 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.568 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.595 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.619 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.666 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.693 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.717 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.740 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.793 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.814 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.836 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.863 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.888 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.908 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.930 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.952 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.971 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.127.992 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.018 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.067 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.093 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.118 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.141 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.176 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.224 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.244 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.266 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.287 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.306 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.326 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.466 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.500 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.525 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.550 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.576 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.600 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.625 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.653 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.677 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.700 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.724 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.772 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.797 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.846 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.873 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.922 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.947 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.972 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.128.997 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.021 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.044 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.068 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.104 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.128 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.154 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.202 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.226 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.299 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.346 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.367 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.392 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.420 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.443 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.470 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.494 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.518 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.543 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.567 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.591 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.617 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.642 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.666 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.689 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.722 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.744 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.768 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.843 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.868 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.891 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.916 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.937 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.961 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.129.984 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.032 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.100 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.124 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.149 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.173 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.221 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.246 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.271 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.294 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.327 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.561 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.670 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.771 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.904 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.930 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.962 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.130.985 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.111 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.217 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.322 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.445 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.549 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.682 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.817 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.842 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.867 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.131.966 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.074 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.099 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.123 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.154 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.191 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.218 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.243 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.365 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.474 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.710 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.735 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.859 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.964 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.132.989 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.130 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.242 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.350 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.454 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.579 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.681 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.708 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.817 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.133.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.142 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.166 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.268 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.372 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.487 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.697 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.803 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.832 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.885 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.957 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.134.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.036 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.060 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.084 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.184 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.288 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.325 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.348 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.368 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.410 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.758 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.865 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.966 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.135.994 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.016 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.132 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.254 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.483 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.695 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.808 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.136.914 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.023 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.127 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.243 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.267 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.288 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.400 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.833 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.137.937 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.070 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.178 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.517 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.622 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.736 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.839 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.138.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.045 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.161 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.185 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.334 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.357 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.380 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.399 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.420 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.438 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.476 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.510 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.544 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.562 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.580 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.623 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.644 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.853 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.139.961 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.062 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.182 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.316 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.427 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.454 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.479 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.606 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.637 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.665 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.690 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.795 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.820 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.841 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.862 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.140.967 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.086 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.191 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.294 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.397 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.496 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.598 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.745 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.856 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.141.978 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.184 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.299 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.326 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.354 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.380 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.484 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.695 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.797 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.142.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.012 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.114 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.221 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.341 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.446 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.562 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.666 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.779 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.806 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.143.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.022 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.057 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.103 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.126 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.151 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.185 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.208 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.310 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.339 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.561 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.661 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.767 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.873 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.144.991 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.095 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.120 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.227 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.334 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.652 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.762 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.875 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.917 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.943 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.969 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.145.994 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.019 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.043 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.064 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.085 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.109 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.134 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.158 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.182 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.204 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.227 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.276 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.301 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.325 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.428 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.539 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.568 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.684 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.893 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.146.978 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.079 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.192 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.307 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.383 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.604 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.707 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.807 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.147.927 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.037 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.153 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.276 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.301 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.402 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.511 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.841 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.866 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.148.966 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.074 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.193 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.397 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.505 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.619 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.719 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.832 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.149.937 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.041 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.283 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.606 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.727 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.834 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.150.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.062 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.166 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.271 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.377 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.490 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.624 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.743 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.846 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.151.951 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.172 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.468 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.589 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.696 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.800 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.152.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.158 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.265 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.369 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.485 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.510 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.742 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.851 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.876 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.933 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.957 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.978 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.153.999 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.018 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.139 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.163 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.186 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.401 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.510 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.615 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.720 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.744 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.765 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.863 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.970 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.154.993 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.108 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.318 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.432 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.455 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.477 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.498 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.602 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.648 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.760 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.783 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.804 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.931 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.951 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.155.971 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.068 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.183 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.293 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.316 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.338 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.443 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.466 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.574 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.691 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.821 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.841 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.939 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.962 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.156.984 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.090 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.205 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.312 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.415 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.624 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.157.957 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.066 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.184 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.291 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.403 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.517 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.625 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.743 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.768 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.919 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.158.941 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.044 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.159 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.265 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.483 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.586 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.707 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.813 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.159.921 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.034 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.059 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.081 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.359 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.484 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.595 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.700 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.809 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.160.928 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.041 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.145 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.258 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.360 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.467 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.581 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.688 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.793 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.161.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.013 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.115 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.230 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.442 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.545 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.769 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.871 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.162.979 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.086 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.110 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.235 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.344 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.457 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.679 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.803 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.163.924 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.030 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.143 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.260 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.371 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.483 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.708 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.816 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.164.919 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.024 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.130 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.258 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.365 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.694 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.165.818 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.041 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.265 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.472 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.601 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.756 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.790 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.861 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.892 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.956 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.166.986 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.013 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.035 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.056 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.075 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.097 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.127 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.151 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.256 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.490 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.598 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.704 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.167.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.020 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.168 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.276 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.505 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.611 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.747 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.769 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.812 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.955 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.168.989 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.019 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.133 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.167 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.199 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.229 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.287 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.315 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.339 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.415 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.444 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.474 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.500 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.578 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.698 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.727 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.838 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.169.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.270 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.414 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.522 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.635 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.738 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.846 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.870 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.170.971 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.084 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.369 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.451 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.557 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.673 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.788 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.171.903 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.015 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.096 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.196 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.429 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.534 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.738 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.879 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.172.985 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.205 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.321 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.686 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.173.901 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.008 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.113 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.264 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.324 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.350 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.376 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.434 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.546 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.659 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.737 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.768 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.793 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.174.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.002 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.118 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.221 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.327 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.457 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.480 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.620 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.645 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.725 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.849 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.175.955 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.068 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.181 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.292 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.402 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.642 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.725 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.754 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.856 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.176.969 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.194 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.305 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.407 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.514 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.598 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.626 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.177.861 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.093 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.231 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.488 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.514 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.539 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.563 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.587 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.611 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.650 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.671 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.691 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.715 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.739 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.762 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.789 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.178.901 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.115 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.234 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.263 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.289 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.394 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.510 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.615 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.723 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.867 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.889 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.910 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.179.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.035 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.139 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.279 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.436 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.462 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.581 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.608 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.714 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.180.941 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.290 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.313 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.421 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.449 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.474 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.526 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.552 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.578 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.604 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.667 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.695 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.771 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.802 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.905 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.928 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.181.951 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.067 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.095 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.121 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.149 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.174 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.199 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.223 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.249 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.274 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.182.298 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.421 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.524 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.610 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.687 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.784 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.862 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.183.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.210 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.464 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.489 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.627 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.706 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.834 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.916 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.184.997 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.185.072 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.185.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.185.433 [opsproto_manager.cc:162]619447 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.185.487 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.185.533 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.111 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.164 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.186 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.268 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.285 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.346 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.362 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.376 [opp_so_manager.cc:56]619447 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.392 [plugin_manager.cc:540]619447 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.406 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.430 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.448 [plugin_manager.cc:547]619447 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.463 [plugin_manager.cc:206]619447 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.186.905 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.186.945 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.186.962 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.186.975 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.187.020 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.187.431 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.464 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.480 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.509 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.187.544 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.594 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.612 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.626 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.187.640 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.673 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.688 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.187.701 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.187.718 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.114 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.149 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.179 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.197 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.237 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.641 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.674 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.691 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.718 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.754 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.801 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.818 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.832 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.847 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.881 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.896 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.188.909 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.188.926 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.189.318 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.351 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.367 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.381 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.189.419 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.189.824 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.858 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.885 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.898 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.189.933 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.978 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.189.996 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.010 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.190.022 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.054 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.069 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.083 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.190.099 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.190.504 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.537 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.553 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.190.568 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.190.601 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.007 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.041 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.070 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.084 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.117 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.164 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.183 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.198 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.212 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.246 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.262 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.276 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.290 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.685 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.718 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.736 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.191.750 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.191.785 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.190 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.237 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.257 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.272 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.310 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.346 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.362 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.376 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.393 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.423 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.438 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.451 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.467 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.480 [plugin_manager.cc:218]619447 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.499 [plugin_manager.cc:422]619447 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.512 [plugin_manager.cc:155]619447 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.543 [plugin_manager.cc:158]619447 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.558 [plugin_manager.cc:426]619447 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.192.950 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.192.985 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.193.000 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.193.016 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.051 [plugin_manager.cc:439]619447 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.430 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.458 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.479 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.588 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.606 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.193.621 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.093 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.196.134 [op_impl_registry_holder_manager.cc:216]619447 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.153 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.237 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.765 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.861 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.880 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.196.894 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.206 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.245 [op_impl_registry_holder_manager.cc:216]619447 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.264 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.303 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.652 [op_impl_space_registry.cc:80]619447 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.671 [op_impl_space_registry.cc:83]619447 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.690 [op_impl_space_registry.cc:80]619447 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.705 [op_impl_space_registry.cc:83]619447 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.718 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.764 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.199.801 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.887 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.915 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.199.928 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.200.694 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.200.725 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.200.817 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.200.835 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.200.849 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.201.478 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.201.502 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.201.544 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.201.578 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.201.662 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.201.680 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.201.693 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.202.291 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.202.316 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.202.404 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.202.432 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.202.446 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.032 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.203.059 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.203.097 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.203.131 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.213 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.230 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.244 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.828 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.203.854 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.939 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.956 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.203.967 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.204.558 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.204.588 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.204.628 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.204.679 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.204.762 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.204.779 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.204.792 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.205.381 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.205.406 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619447,python3.9):2025-04-03-09:35:44.205.491 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.205.509 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.205.522 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.206.103 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.206.128 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.206.166 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.206.200 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.206.234 [opp_so_manager.cc:97]619447 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.206.305 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.206.351 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.206.378 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.206.391 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.485 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.529 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.550 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.571 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.594 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.613 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.650 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.689 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.708 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.727 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.745 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.783 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.801 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.820 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.838 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.857 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.877 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.894 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.912 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.930 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.950 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.240.970 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.003 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.021 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.039 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.058 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.078 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.117 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.155 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.176 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.195 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.216 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.235 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.252 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.273 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.292 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.310 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.327 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.347 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.366 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.406 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.425 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.442 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.476 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.558 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.576 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.596 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.616 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.636 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.655 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.672 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.691 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.710 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.729 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.766 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.785 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.804 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.862 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.880 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.899 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.918 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.934 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.953 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.241.994 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.020 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.041 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.059 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.075 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.095 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.112 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.130 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.149 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.170 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.190 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.230 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.249 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.266 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.285 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.302 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.322 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.360 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.396 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.414 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.435 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.455 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.476 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.493 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.510 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.537 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.554 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.573 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.611 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.647 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.668 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.686 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.707 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.728 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.749 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.770 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.811 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.829 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.850 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.869 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.887 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.947 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.968 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.242.988 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.005 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.023 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.054 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.096 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.115 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.136 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.155 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.176 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.193 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.211 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.231 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.252 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.317 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.426 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.450 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.471 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.491 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.511 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.554 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.573 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.593 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.614 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.635 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.656 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.684 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.704 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.724 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.744 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.784 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.802 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.822 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.841 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.866 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.889 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.911 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.928 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.949 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.969 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.243.989 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.010 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.065 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.085 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.109 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.129 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.149 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.198 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.221 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.242 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.270 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.292 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.313 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.335 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.357 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.377 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.398 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.418 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.437 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.500 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.539 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.562 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.608 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.649 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.690 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.711 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.737 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.758 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.777 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.864 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.883 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.903 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.923 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.944 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.964 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.244.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.004 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.025 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.125 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.221 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.320 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.415 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.514 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.539 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.562 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.583 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.680 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.784 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.883 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.245.976 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.072 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.274 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.298 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.329 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.349 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.440 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.535 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.557 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.580 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.599 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.621 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.642 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.664 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.761 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.856 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.246.963 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.059 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.083 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.269 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.319 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.341 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.432 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.631 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.724 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.855 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.947 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.247.972 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.069 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.176 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.274 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.371 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.486 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.582 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.679 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.776 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.877 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.970 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.248.994 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.017 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.057 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.081 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.125 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.146 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.168 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.298 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.404 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.428 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.449 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.471 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.512 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.610 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.706 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.821 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.249.914 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.011 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.035 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.058 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.156 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.351 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.444 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.540 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.635 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.736 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.830 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.250.924 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.021 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.123 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.148 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.184 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.283 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.402 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.585 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.679 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.775 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.871 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.251.988 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.086 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.289 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.316 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.410 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.501 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.593 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.687 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.784 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.885 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.909 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.252.930 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.020 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.044 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.068 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.106 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.127 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.150 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.171 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.192 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.213 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.232 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.269 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.289 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.311 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.352 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.378 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.472 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.564 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.658 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.754 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.851 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.253.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.065 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.111 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.204 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.238 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.280 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.373 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.396 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.418 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.439 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.632 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.724 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.818 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.254.913 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.009 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.211 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.303 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.488 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.582 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.678 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.701 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.726 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.749 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.839 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.255.933 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.133 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.240 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.342 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.435 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.527 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.629 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.719 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.821 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.256.917 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.009 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.033 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.127 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.220 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.244 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.267 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.292 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.313 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.336 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.357 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.378 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.464 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.585 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.682 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.774 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.872 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.257.978 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.172 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.195 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.286 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.383 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.473 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.569 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.857 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.879 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.900 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.921 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.960 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.258.980 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.000 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.021 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.062 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.108 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.129 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.148 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.184 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.205 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.227 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.250 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.347 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.446 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.470 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.565 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.661 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.752 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.259.917 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.011 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.186 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.285 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.483 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.576 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.670 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.762 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.953 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.260.977 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.069 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.163 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.370 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.468 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.491 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.586 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.680 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.777 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.873 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.261.966 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.059 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.153 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.248 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.345 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.439 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.540 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.632 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.757 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.852 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.262.947 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.044 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.141 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.239 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.544 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.636 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.729 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.828 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.263.952 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.044 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.141 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.245 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.341 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.434 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.536 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.630 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.815 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.264.905 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.003 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.191 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.283 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.477 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.500 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.600 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.691 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.784 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.816 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.839 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.859 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.877 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.919 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.940 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.265.961 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.053 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.096 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.186 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.281 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.376 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.470 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.567 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.617 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.710 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.822 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.266.916 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.009 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.195 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.229 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.277 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.367 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.390 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.412 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.524 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.546 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.642 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.668 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.690 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.711 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.801 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.894 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.267.987 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.010 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.122 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.145 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.254 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.349 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.442 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.467 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.489 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.589 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.612 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.632 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.822 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.268.918 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.011 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.204 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.296 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.392 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.484 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.578 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.672 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.770 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.866 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.269.976 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.083 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.183 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.208 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.229 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.346 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.366 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.462 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.564 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.670 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.764 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.864 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.270.959 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.155 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.349 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.397 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.419 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.515 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.617 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.722 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.818 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.271.921 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.018 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.115 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.227 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.335 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.435 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.537 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.636 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.732 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.828 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.272.941 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.138 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.237 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.336 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.437 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.633 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.727 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.827 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.273.924 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.035 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.057 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.155 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.256 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.358 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.557 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.662 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.761 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.858 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.274.956 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.053 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.154 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.256 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.356 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.555 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.659 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.757 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.855 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.275.954 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.169 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.278 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.478 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.573 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.675 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.782 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.897 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.926 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.950 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.276.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.000 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.023 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.069 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.094 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.114 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.168 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.189 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.213 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.233 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.330 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.432 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.524 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.621 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.822 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.277.920 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.002 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.126 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.222 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.326 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.428 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.523 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.623 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.646 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.690 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.712 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.839 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.868 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.904 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.926 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.950 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.278.980 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.004 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.027 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.074 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.097 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.127 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.147 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.172 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.224 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.248 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.271 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.352 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.480 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.575 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.670 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.767 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.864 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.279.970 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.075 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.309 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.417 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.544 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.849 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.280.958 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.028 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.106 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.217 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.322 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.538 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.638 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.718 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.803 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.886 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.281.990 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.095 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.200 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.420 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.525 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.631 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.842 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.282.945 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.157 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.263 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.365 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.466 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.565 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.693 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.717 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.738 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.759 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.780 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.827 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.283.925 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.021 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.092 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.116 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.245 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.355 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.467 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.576 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.679 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.778 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.801 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.824 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.284.983 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.007 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.086 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.188 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.288 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.488 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.589 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.704 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.807 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.285.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.005 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.101 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.203 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.321 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.424 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.526 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.628 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.819 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.842 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.286.944 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.045 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.144 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.241 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.343 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.366 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.411 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.454 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.477 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.497 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.518 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.538 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.558 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.580 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.600 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.621 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.731 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.832 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.937 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.962 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.287.983 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.087 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.212 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.531 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.556 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.578 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.599 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.624 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.821 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.288.919 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.058 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.197 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.298 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.398 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.507 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.616 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.725 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.749 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.851 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.873 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.289.980 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.013 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.034 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.056 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.077 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.098 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.118 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.139 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.159 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.182 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.206 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.226 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.247 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.267 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.284 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.407 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.430 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.530 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.553 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.574 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.596 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.618 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.638 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.659 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.678 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.699 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.850 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.290.923 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.117 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.192 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.264 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.579 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.733 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.814 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.888 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.291.962 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.031 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.145 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.270 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.448 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.519 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.608 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.292.882 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.302.040 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.306.729 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.328.802 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.329.017 [plugin_manager.cc:554]619447 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619447,python3.9):2025-04-03-09:35:44.329.041 [plugin_manager.cc:134]619447 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.329.103 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.329.122 [plugin_manager.cc:145]619447 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.329.136 [plugin_manager.cc:557]619447 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.329.151 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.329.179 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.037 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.116 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.140 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.226 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.243 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.298 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.315 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.327 [opp_so_manager.cc:66]619447 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.342 [plugin_manager.cc:616]619447 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.357 [plugin_manager.cc:108]619447 GetOppPath:Enter get opp path schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.381 [plugin_manager.cc:117]619447 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.401 [plugin_manager.cc:624]619447 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.419 [plugin_manager.cc:206]619447 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.848 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.330.887 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.330.905 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.330.918 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.330.967 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.331.383 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.418 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.433 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.446 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.331.482 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.532 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.549 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.331.562 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.331.577 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.331.595 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.331.991 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.023 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.039 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.069 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.332.104 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.332.514 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.549 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.567 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.584 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.332.620 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.659 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.675 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.332.692 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.332.709 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.332.724 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.115 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.148 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.164 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.176 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.212 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.606 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.650 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.666 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.680 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.713 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.746 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.761 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.333.775 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.791 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.333.806 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.193 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.225 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.241 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.255 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.291 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.684 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.714 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.732 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.744 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.780 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.835 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.851 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.334.863 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.877 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.334.894 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.287 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.319 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.334 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.348 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.382 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.779 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.810 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.825 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.838 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.872 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.909 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.924 [plugin_manager.cc:1026]619447 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.335.952 [plugin_manager.cc:322]619447 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.969 [plugin_manager.cc:212]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.986 [plugin_manager.cc:215]619447 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.335.998 [plugin_manager.cc:218]619447 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.020 [plugin_manager.cc:422]619447 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.032 [plugin_manager.cc:155]619447 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.054 [plugin_manager.cc:158]619447 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.068 [plugin_manager.cc:426]619447 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.465 [plugin_manager.cc:250]619447 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.336.503 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.336.521 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.336.536 [plugin_manager.cc:1063]619447 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.573 [plugin_manager.cc:439]619447 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.962 [plugin_manager.cc:996]619447 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619447,python3.9):2025-04-03-09:35:44.336.991 [plugin_manager.cc:1011]619447 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619447,python3.9):2025-04-03-09:35:44.337.011 [plugin_manager.cc:1015]619447 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.337.070 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.337.181 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.337.198 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.337.212 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.071 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff40a192c0, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.114 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.137 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.158 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.177 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.475 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff40a1d978, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.498 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.517 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.534 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.552 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.825 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.339.860 [op_impl_registry_holder_manager.cc:216]619447 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.888 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.339.922 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.340.198 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.340.257 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.340.373 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.340.391 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.340.403 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.341.954 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.341.989 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.009 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.026 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.043 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.322 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff409ca9b0, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.345 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.365 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.382 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.397 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.673 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff409cdda0, registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.705 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.721 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.739 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.342.757 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.002 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.029 [op_impl_registry_holder_manager.cc:216]619447 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.046 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.082 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.414 [op_impl_space_registry.cc:84]619447 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.431 [op_impl_space_registry.cc:85]619447 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.444 [op_impl_space_registry.cc:86]619447 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.456 [op_impl_space_registry.cc:87]619447 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.468 [op_impl_space_registry.cc:93]619447 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.487 [op_impl_space_registry.cc:84]619447 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.501 [op_impl_space_registry.cc:85]619447 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.515 [op_impl_space_registry.cc:86]619447 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.529 [op_impl_space_registry.cc:87]619447 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.543 [op_impl_space_registry.cc:93]619447 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.556 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.612 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.651 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.343.688 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.784 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.801 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.343.813 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.344.384 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.344.413 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.344.459 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.344.556 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.344.573 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.344.586 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.345.236 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.345.263 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.345.306 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.345.355 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.345.391 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.345.484 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.345.501 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.345.515 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.345.972 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.345.996 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.346.035 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.130 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.146 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.157 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.738 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.346.764 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.346.806 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.346.841 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.346.885 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.975 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.346.992 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.347.003 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.347.440 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.347.463 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.347.502 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.347.594 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.347.610 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.347.622 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.348.202 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.233 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.278 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.313 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.349 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.348.451 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.348.467 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.348.480 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.348.917 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.940 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.348.979 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.072 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.088 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.099 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.674 [op_impl_space_registry.cc:240]619447 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.701 [opp_so_manager.cc:48]619447 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.742 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.777 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.814 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.839 [opp_so_manager.cc:126]619447 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.349.883 [opp_so_manager.cc:123]619447 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.964 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.349.980 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.350.024 [plugin_manager.cc:1126]619447 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.350.040 [plugin_manager.cc:1130]619447 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.350.053 [opp_so_manager.cc:45]619447 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.363.469 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.364.291 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.364.349 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.364.378 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.367.084 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.388.261 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.388.263 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.388.298 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.388.309 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.317 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.328 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.345 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.358 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.388.382 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.388.382 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.388.407 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.388.421 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.458 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.388.458 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.487.778 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.340 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.427 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.506 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.581 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.648 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.751 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.818 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.842 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.861 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.879 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.488.992 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.066 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.133 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.208 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.346 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.367 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.434 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.455 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.475 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.546 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.634 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.701 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.775 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.844 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.909 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.489.980 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.049 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.125 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.192 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.259 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.328 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.394 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.461 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.528 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.603 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.490.676 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.496.524 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.496.676 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.496.792 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.086 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.214 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.324 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.602 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.713 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.497.893 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.041 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.279 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.418 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.536 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.654 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.679 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.838 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.498.955 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.102 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.219 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.334 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.358 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.476 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.592 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.756 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.878 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.499.989 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.105 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.257 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.496 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.608 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.717 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.858 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.500.882 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.100 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.222 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.365 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.675 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xffff0a90d360, registered count:5 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.704 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.726 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.747 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.768 [op_check.cc:71]619447 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.790 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.501.920 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.034 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.230 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.516 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.544 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.649 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.675 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.781 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.502.919 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.032 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.147 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.257 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.584 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.612 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.715 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.503.829 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.019 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.142 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.261 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.382 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.509 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.724 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.504.887 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.036 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.185 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.300 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.416 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.505.759 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.032 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.205 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.231 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xffff0a997e08, registered count:7 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.391 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.411 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xffff0a9a0298, registered count:7 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.548 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.661 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.826 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.506.942 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.169 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.275 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.512 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.619 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.645 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.753 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.869 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.507.979 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.094 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.218 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.338 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.449 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.566 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.711 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.736 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.760 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.782 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.508.935 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.164 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.278 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.506 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.528 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.564 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.781 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.509.887 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.084 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.193 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.305 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.430 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.548 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.657 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.772 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.797 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.510.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.013 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.332 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.447 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.555 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.669 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.776 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.511.897 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.512.007 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.512.128 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.512.347 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.512.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.512.585 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.219 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.272 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.437 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.565 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.672 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.793 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.515.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.012 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.121 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.254 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.386 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.415 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.531 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.555 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.667 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.691 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.797 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.516.907 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.024 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.151 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.665 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.822 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.517.932 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.518.046 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.518.161 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.518.828 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.519.644 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.519.979 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.520.457 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.071 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.192 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.320 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.467 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.578 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.521.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.524.076 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.524.945 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.525.107 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.525.266 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.526.270 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.526.425 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.526.579 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.167 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.418 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.526 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.643 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.751 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.862 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.527.980 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.099 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.228 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.508 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.628 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.816 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.528.934 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.042 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.158 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.324 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.496 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.609 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.719 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.743 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.529.946 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.053 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.162 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.336 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.623 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.737 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.858 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.967 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.530.991 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.100 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.125 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.246 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.350 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.376 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.596 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.714 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.531.906 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.020 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.219 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.384 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.565 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.732 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.532.911 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.106 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.220 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.334 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.447 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.561 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.781 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.533.891 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.000 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.106 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.213 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.337 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.451 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.475 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.594 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.534.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.535.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.538.613 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.538.826 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.539.033 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.539.367 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.539.531 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.539.685 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.540.531 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.540.696 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.540.830 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.540.985 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.136 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.265 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.429 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.585 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.742 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.873 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.541.982 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.076 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.410 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.436 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.528 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.549 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.570 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.591 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.614 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.636 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.655 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.677 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.698 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.721 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.741 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.763 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.785 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.809 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.829 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.869 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.901 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.922 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.943 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.963 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.542.986 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.007 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.029 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.051 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.075 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.095 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.117 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.137 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.161 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.183 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.204 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.226 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.251 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.271 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.292 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.312 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.333 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.356 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.379 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.400 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.424 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.480 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.502 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.521 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.542 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.563 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.583 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.604 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.626 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.779 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.543.929 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.088 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.116 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.140 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.171 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.198 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.352 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.470 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.582 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.686 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.790 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.544.902 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.174 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.333 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.450 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.571 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.596 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.620 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.755 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.781 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.545.895 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.057 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.210 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.357 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.467 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.663 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.775 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.546.957 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.065 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.187 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.212 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.239 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.391 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.548 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.697 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.547.848 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.195 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.320 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.492 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.651 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.823 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.548.954 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.064 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.295 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.405 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.556 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.710 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.820 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.549.944 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.550.054 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.550.175 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.550.282 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.550.732 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.168 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.294 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.434 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.547 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.662 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.773 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.551.894 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.003 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.120 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.242 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.356 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.518 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.637 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.755 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.552.940 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.050 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.167 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.279 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.463 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.644 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.762 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.883 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.553.995 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.109 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.209 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.323 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.435 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.585 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.696 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.554.854 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.555.132 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.555.314 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.555.477 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.555.667 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.555.885 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.084 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.321 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.346 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.368 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.388 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.415 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.438 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.458 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.556.479 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.557.243 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.557.651 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.558.064 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.558.258 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.558.459 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.558.879 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.558.998 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.559.166 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.561.718 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.563.553 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.563.683 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.563.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.563.915 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.213 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.377 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.574 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.909 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.939 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.963 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.564.986 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.012 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.198 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.223 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.384 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.543 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.715 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.565.882 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.062 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.091 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.244 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.421 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.584 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.750 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.566.921 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.073 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.099 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.257 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.444 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.480 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.588 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.709 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.567.734 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.117 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.145 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.281 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.401 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.522 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.641 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.746 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.568.860 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.569.043 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.569.241 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.569.628 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.570.003 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.570.197 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.570.533 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.570.718 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.570.913 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.038 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.157 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.309 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.335 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.361 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.571.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.572.523 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.572.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.150 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.303 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.453 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.482 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.638 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.790 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.951 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.573.976 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.574.132 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.574.288 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.574.433 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.574.599 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.574.748 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.575.134 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.575.489 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.575.730 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.575.910 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.578.876 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.579.265 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.579.791 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.579.940 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.088 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.206 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.374 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.514 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.541 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.653 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.680 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.580.988 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.097 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.245 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.396 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.504 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.798 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.581.975 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.120 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.223 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.325 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.469 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.631 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.792 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.819 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.582.974 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.583.000 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.583.139 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.583.307 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.584.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.590.395 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.590.451 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.592.995 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.593.247 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.593.516 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.593.828 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.023 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.214 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.364 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.503 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.681 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.814 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.594.900 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.595.133 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.595.244 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.595.388 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.595.834 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.595.979 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.123 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.153 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.281 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.390 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.558 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.728 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.596.908 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.055 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.169 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.273 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.297 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.483 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.657 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.799 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.597.903 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.052 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.080 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.201 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.303 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.331 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.355 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.490 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.672 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.756 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.598.972 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.599.152 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.599.177 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.599.201 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.599.380 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.599.408 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.276 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.363 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.387 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.459 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.554 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.576 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.647 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.814 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.834 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.954 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.974 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.602.994 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.603.012 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.603.176 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.603.195 [tuning_tiling_registry.cc:30]619447 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.604.426 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xffff0b2dc0b0, registered count:7 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.604.455 [op_check.cc:24]619447 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xffff0b2dc310, registered count:5 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.604.853 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.605.349 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.607.024 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.607.199 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.607.278 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.607.651 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.608.099 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.638.893 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.639.039 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.639.229 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.639.311 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.639.466 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.639.610 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.640.301 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.640.389 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.640.665 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.640.747 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.640.898 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.641.135 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.641.313 [op_impl_registry.cc:122]619447 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619447,python3.9):2025-04-03-09:35:44.642.012 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.647.698 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619447,python3.9):2025-04-03-09:35:44.649.838 [op_impl_registry_holder_manager.cc:208]619447 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.723.680 [op_impl_space_registry.cc:109]619447 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.725.034 [op_impl_space_registry.cc:84]619447 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.725.067 [op_impl_space_registry.cc:85]619447 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.725.082 [op_impl_space_registry.cc:86]619447 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.725.097 [op_impl_space_registry.cc:87]619447 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.725.112 [op_impl_space_registry.cc:93]619447 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.126 [op_impl_space_registry.cc:273]619447 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.422 [plugin_manager.cc:564]619447 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.451 [plugin_manager.cc:134]619447 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.726.513 [file_utils.cc:47]619447 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.726.531 [plugin_manager.cc:145]619447 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619447,python3.9):2025-04-03-09:35:44.726.542 [plugin_manager.cc:567]619447 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.634 [mem_manager.cc:31]619447 Initialize:MemManager Initialize. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.668 [rts_caching_mem_allocator.cc:131]619447 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.687 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.700 [graph_mem_allocator.cc:17]619447 Initialize:MemoryAllocator::Initialize. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.717 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.731 [graph_mem_allocator.cc:17]619447 Initialize:MemoryAllocator::Initialize. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.743 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.755 [caching_allocator.cc:391]619447 FreeBlocks:Free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.768 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.786 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.800 [caching_allocator.cc:391]619447 FreeBlocks:Free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.809 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.823 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.838 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.850 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.865 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.878 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.892 [session_scope_mem_allocator.cc:73]619447 FreeAllMemory:Free all memory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.904 [mem_manager.cc:102]619447 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.918 [session_scope_mem_allocator.cc:73]619447 FreeAllMemory:Free all memory [INFO] GE(619447,python3.9):2025-04-03-09:35:44.726.969 [dump_manager.cc:152]619447 Init:exception dump is not enabled [INFO] GE(619447,python3.9):2025-04-03-09:35:44.727.027 [ge_executor.cc:315]619447 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.064 [acl.cpp:321]619447 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.081 [dump.cpp:841]619447 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.263 [dump.cpp:759]619447 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.277 [dump.cpp:594]619447 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.288 [dump.cpp:224]619447 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.320 [dump.cpp:703]619447 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.340 [dump.cpp:771]619447 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.727.355 [dump.cpp:703]619447 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.427 [adx_datadump_server.cpp:77][tid:619447] start to do dump init [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.457 [adx_dump_record.cpp:210][tid:619447] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.475 [adx_dump_record.cpp:214][tid:619447] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.676 [adx_server_manager.cpp:149][tid:619672] server manager add component (10) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.690 [adx_dump_record.cpp:627][tid:619673] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.705 [adx_server_manager.cpp:165][tid:619672] server manager components init successfully [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.821 [adx_server_manager.cpp:203][tid:619674] Run Server(0) Process [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.877 [adx_server_manager.cpp:369][tid:619674] device up 0 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.727.897 [hdc_comm_opt.cpp:59][tid:619674] open device[0] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.076 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.096 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.114 [hdc_api.cpp:119][tid:619674] logDevId 0 create HDC server successfully [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.154 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.186 [adx_server_manager.cpp:369][tid:619674] device up 1 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.196 [hdc_comm_opt.cpp:59][tid:619674] open device[1] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.357 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.373 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.385 [hdc_api.cpp:119][tid:619674] logDevId 1 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.398 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.415 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.429 [adx_server_manager.cpp:369][tid:619674] device up 2 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.439 [hdc_comm_opt.cpp:59][tid:619674] open device[2] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.567 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.583 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.599 [hdc_api.cpp:119][tid:619674] logDevId 2 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.609 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.623 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.637 [adx_server_manager.cpp:369][tid:619674] device up 3 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.660 [hdc_comm_opt.cpp:59][tid:619674] open device[3] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.817 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.834 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.847 [hdc_api.cpp:119][tid:619674] logDevId 3 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.862 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.874 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.888 [adx_server_manager.cpp:369][tid:619674] device up 4 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.728.898 [hdc_comm_opt.cpp:59][tid:619674] open device[4] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.728.995 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.011 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.027 [hdc_api.cpp:119][tid:619674] logDevId 4 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.038 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.052 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.066 [adx_server_manager.cpp:369][tid:619674] device up 5 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.076 [hdc_comm_opt.cpp:59][tid:619674] open device[5] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.213 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.228 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.244 [hdc_api.cpp:119][tid:619674] logDevId 5 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.255 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.269 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.282 [adx_server_manager.cpp:369][tid:619674] device up 6 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.292 [hdc_comm_opt.cpp:59][tid:619674] open device[6] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.389 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.405 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.417 [hdc_api.cpp:119][tid:619674] logDevId 6 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.429 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.442 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.456 [adx_server_manager.cpp:369][tid:619674] device up 7 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.466 [hdc_comm_opt.cpp:59][tid:619674] open device[7] server[13] [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.615 [drv_log_user.c:623][ascend][curpid:619447,619674][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.729.632 [hdc_server.c:363][ascend][curpid:619447,619674][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.649 [hdc_api.cpp:119][tid:619674] logDevId 7 create HDC server successfully [WARNING] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.661 [adx_hdc_epoll.cpp:34][tid:619674] hdc epoll has been created. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.729.675 [adx_server_manager.cpp:111][tid:619674] create server info [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.730.780 [adx_server_manager.cpp:457][tid:619447] The server is initialized after waiting 3 times. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.730.803 [ge_executor.cc:1349]619447 SetDump:Start to set dump config [INFO] GE(619447,python3.9):2025-04-03-09:35:44.730.824 [dump_manager.cc:201]619447 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.730.861 [dump_manager.cc:261]619447 SetExceptionDump:Enable brief exception dump. [INFO] GE(619447,python3.9):2025-04-03-09:35:44.730.894 [ge_executor.cc:1355]619447 SetDump:Set dump config successfully [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.730.929 [dump_manager.cpp:146][tid:619447] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:44.732.461 [dump_manager.cpp:301][tid:619447] Register exception callback, registered: 1 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.484 [dump.cpp:825]619447 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.508 [acl.cpp:327]619447 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.523 [acl.cpp:329]619447 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.536 [acl_resource_manager.cpp:248]619447 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.635 [acl_resource_manager.cpp:275]619447 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.652 [acl_resource_manager.cpp:278]619447 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.670 [acl.cpp:335]619447 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.688 [acl.cpp:336]619447 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.752 [json_parser.cpp:260]619447 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.770 [acl.cpp:343]619447 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:44.732.792 [prof_atls_plugin.cpp:210] (tid:619447) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.808 [acl.cpp:351]619447 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.821 [profiling.cpp:141]619447 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.837 [profiling.cpp:135]619447 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.900 [profiling.cpp:167]619447 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619447.json], errorCode = 0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.928 [profiling.cpp:172]619447 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619447.json] is empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.940 [profiling.cpp:176]619447 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.955 [profiling.cpp:179]619447 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.732.966 [profiling.cpp:92]619447 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:44.733.378 [ai_drv_dev_api.cpp:316] >>> (tid:619447) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:44.733.508 [msprof_callback_impl.cpp:132] >>> (tid:619447) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.527 [profiling.cpp:122]619447 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.541 [profiling.cpp:186]619447 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.556 [hash_utils.cpp:108]619447 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619447.json [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.599 [hash_utils.cpp:128]619447 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619447.json, hash: 3917491750093904922 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.616 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.651 [acl.cpp:280]619447 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.666 [acl.cpp:385]619447 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.733.685 [runtime.cc:5394] 619447 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.733.705 [api_impl.cc:7407] 619447 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.733.739 [api_impl.cc:4211] 619447 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.733.758 [acl.cpp:420]619447 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.733.963 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.734.133 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.734.207 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.734.235 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.740165] [info] [619447] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist mki_log delete old file:/home/jenkins/atb/log/atb_1952516_20250328090703.log [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:44.740.845 [prof_atls_plugin.cpp:210] (tid:619447) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.741.748 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.741.775 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.741.852 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 7, rank size = 8. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.741.870 [device.cpp:143]619447 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.897 [api_impl.cc:2999] 619447 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.915 [device.cc:306] 619447 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.931 [runtime.cc:3393] 619447 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.963 [runtime.cc:3560] 619447 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.979 [api_impl.cc:2987] 619447 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.741.995 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.742.014 [api_impl.cc:2992] 619447 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.742.033 [api_impl.cc:3025] 619447 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.742.048 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.742.060 [acl.cpp:121]619447 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.742.096 [device.cpp:154]619447 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.742.146 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.742.165 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is _npu_log, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.742.187 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.742.203 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.742.216 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.742.251 [queue_client.c:151][ascend][curpid:619447,619447][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.742.420 [queue_client.c:305][ascend][curpid:619447,619447][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=7; vfid=0; hostpid=619447; master_pid=619447; proc_type=0; devpid=30963). [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.746.391 [queue_client.c:362][ascend][curpid:619447,619447][drv][queuemng][QueueInit]QueueInit device. (devid=7; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.746.573 [npu_driver.cc:4114] 619447 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.746.605 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.746.623 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=_npu_log. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.746.929 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.746.949 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.086 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.114 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.143 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.747.148 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.162 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.175 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.196 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.210 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.222 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.275 [npu_driver.cc:4087] 619447 MemQueueInit: repeated init [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.291 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.308 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=ms_tensor_dump. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.747.495 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.514 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.615 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.642 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.747.656 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.661 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.678 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.682 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.697 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.722 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.737 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.770 [npu_driver.cc:4087] 619447 MemQueueInit: repeated init [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.786 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.747.797 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=ms_tensor_summary. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.747.963 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.747.982 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.151 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.748.199 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.200 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.221 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.235 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.243 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.250 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.257 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.270 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.305 [npu_driver.cc:4087] 619447 MemQueueInit: repeated init [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.321 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.339 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=ms_image_summary. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.748.488 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.507 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.607 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.636 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.748.637 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.691 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.697 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.709 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.716 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.728 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.741 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.779 [npu_driver.cc:4087] 619447 MemQueueInit: repeated init [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.794 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.748.810 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=ms_scalar_summary. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.748.956 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.748.975 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.066 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.749.093 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.098 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.112 [tensor_data_transfer.cpp:898]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.122 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.132 [api_impl.cc:6077] 619447 MemQueueInit: Start to init queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.137 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.145 [npu_driver.cc:4076] 619447 MemQueueInit: init mem queue, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.157 [npu_driver.cc:4026] 619447 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.193 [npu_driver.cc:4087] 619447 MemQueueInit: repeated init [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.208 [api_impl.cc:6040] 619447 MemQueueCreate: Start to create queue on devId 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.220 [npu_driver.cc:3931] 619447 MemQueueCreate: create mem queue, deviceId=7, name=ms_histogram_summary. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.749.366 [queue_interface.c:182][ascend][curpid:619447,619447][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.393 [tensor_data_transfer.cpp:940]619447 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 7, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.560 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.749.600 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.749.600 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.633 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.749.650 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.749.775 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.749.811 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.830 [plugin_manager.cc:25] [619447]hcom ops plugin init start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.868 [plugin_manager.cc:38] [619447]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.889 [plugin_manager.cc:42] [619447]hcom running normal mode. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.933 [hcom_plugin.cc:50] [619447]Initialize start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.948 [hcom_plugin.cc:52] [619447]initializeCount_ : 1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.749.993 [hcom_plugin.cc:60] [619447]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.024 [hcom_plugin.cc:63] [619447]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.040 [hcom_plugin.cc:329] [619447]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.057 [hcom_plugin.cc:281] [619447]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:44.750.081 [hcom_plugin.cc:217] [619447]option profiling mode is not found. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.099 [hcom_plugin.cc:164] [619447][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.116 [hcom_plugin.cc:180] [619447][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.132 [hcom_plugin.cc:244] [619447]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.263 [hcom_plugin.cc:82] [619447]hccl ops plugin init success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.279 [plugin_manager.cc:48] [619447]hcom ops plugin init end. ret[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.294 [hcom_plugin.cc:126] [619447]get hccl kernel info store start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.322 [hcom_plugin.cc:132] [619447]get hccl kernel info store finished. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.750.351 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.750.377 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.407 [hcom_executor.cc:29] [619447]Hcom Excutor Initialize start. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.457 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.477 [adapter_rts.cc:234] [619447][hrtGetDevice]deviceLogicId[7] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.512 [api_c_soc.cc:70] 619447 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.563 [hcom_common.cc:85] [619447][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.592 [hcom_common.cc:88] [619447][HcomGetCurHcomCtx] is no Used deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.618 [hcom.cc:486] [619447]Get Hcom Init Status: [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.635 [hcom_common.cc:85] [619447][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.750.653 [hcom_rpc.cc:1019] [619447]Get Hcom Init Status: [0] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:44.750.678 [hcom_base_funs.cc:36] [619447][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:44.750.711 [hcom_executor.cc:121] [619447][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.773 [api_impl.cc:1120] 619447 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.801 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.881 [stream.cc:2246] 619447 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.966 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.750.983 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.560 [npu_driver.cc:6125] 619447 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.614 [npu_driver.cc:6137] 619447 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=342, streamId=4, logicCq=342, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.642 [npu_driver.cc:6189] 619447 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=4, logicCq=342. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.664 [npu_driver.cc:5916] 619447 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f4087000, addr=dffff4087000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.680 [stream.cc:653] 619447 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.751.759 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.751.778 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.127 [stream.cc:291] 619447 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.751 [stream.cc:688] 619447 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=7, streamResId=65535 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.752.772 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.752.788 [acl_resource_manager.cpp:816]619447 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.816 [api_impl.cc:1162] 619447 StreamCreate: Succ, flags=1536, stream_id=4, context=aaab0ca695c0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.752.854 [stream.cc:73] [619447]Construct stream by stream type success, ptr[0xaaab17d90120] ctx[0xaaab0ca695c0], stream id[4], cqId[4], logicCqid[342] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.893 [api_impl.cc:1120] 619447 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.917 [npu_driver.cc:404] 619447 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.945 [stream.cc:2246] 619447 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.982 [stream_sqcq_manage.cc:147] 619447 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.752.999 [npu_driver.cc:882] 619447 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.378 [npu_driver.cc:6125] 619447 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.421 [npu_driver.cc:6137] 619447 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=343, streamId=5, logicCq=343, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.443 [npu_driver.cc:6189] 619447 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=5, logicCq=343. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.461 [npu_driver.cc:5916] 619447 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f4066000, addr=dffff4066000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.477 [stream.cc:653] 619447 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619447,python3.9):2025-04-03-09:35:44.753.518 [devmm_svm.c:1972][ascend][curpid:619447,619447][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.536 [npu_driver.cc:6080] 619447 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.753.828 [stream.cc:291] 619447 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.467 [stream.cc:688] 619447 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=7, streamResId=65535 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.754.486 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.754.502 [acl_resource_manager.cpp:816]619447 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.518 [api_impl.cc:1162] 619447 StreamCreate: Succ, flags=1536, stream_id=5, context=aaab0ca695c0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.754.540 [stream.cc:73] [619447]Construct stream by stream type success, ptr[0xaaab17e0c180] ctx[0xaaab0ca695c0], stream id[5], cqId[5], logicCqid[343] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:44.754.669 [sal.cc:619] [619702]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.754.696 [hcom_executor.cc:587] [619702]MessageProcessThreadLoop start queType:[0] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:44.754.716 [sal.cc:619] [619703]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.723 [api_impl.cc:2999] 619702 SetDevice: device_id=7, deviceMode=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.754.746 [hcom_executor.cc:587] [619703]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.750 [device.cc:306] 619702 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.765 [api_impl.cc:2999] 619704 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.770 [api_impl.cc:2999] 619703 SetDevice: device_id=7, deviceMode=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:44.754.777 [hcom_executor.cc:34] [619447][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.781 [runtime.cc:3393] 619702 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.788 [device.cc:306] 619704 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.812 [runtime.cc:3560] 619702 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.807 [api_impl.cc:2999] 619705 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.821 [runtime.cc:3393] 619704 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.823 [device.cc:306] 619703 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.825 [api_impl.cc:2987] 619702 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.754.828 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.842 [runtime.cc:3560] 619704 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.855 [runtime.cc:3393] 619703 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.856 [runtime.cc:5239] 619702 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.856 [device.cc:306] 619705 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.871 [api_impl.cc:2987] 619704 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.754.874 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.878 [runtime.cc:3560] 619703 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.889 [api_impl.cc:2992] 619702 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.890 [runtime.cc:3393] 619705 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.895 [runtime.cc:5239] 619704 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.754.906 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.906 [api_impl.cc:2987] 619703 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.917 [runtime.cc:3560] 619705 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:44.754.918 [awatchdog_core.c:175](tid:619702) create watchdog for id : 65543, tid : 619702, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.925 [api_impl.cc:2992] 619704 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.934 [runtime.cc:5239] 619703 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.939 [api_impl.cc:2987] 619705 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.754.942 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.946 [api_impl.cc:3025] 619702 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:44.754.953 [awatchdog_core.c:175](tid:619704) create watchdog for id : 65543, tid : 619704, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.976 [api_impl.cc:2992] 619703 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.977 [runtime.cc:5239] 619705 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.754.998 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.754.997 [api_impl.cc:3025] 619704 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:44.755.008 [awatchdog_core.c:175](tid:619703) create watchdog for id : 65543, tid : 619703, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.011 [api_impl.cc:2992] 619705 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.027 [api_impl.cc:3025] 619703 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:44.755.035 [awatchdog_core.c:175](tid:619705) create watchdog for id : 65543, tid : 619705, timeout : 300s [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.045 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1152.2 msec. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.051 [api_impl.cc:3025] 619705 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.080 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [WARNING] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.105 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.165 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.018 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.219 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.260 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.285 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.095 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.398 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:44.755.439 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:44.755.514 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:44.755.548 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.755.594 [device.cpp:143]619706 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.625 [api_impl.cc:2999] 619706 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.642 [device.cc:306] 619706 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.655 [runtime.cc:3393] 619706 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.670 [runtime.cc:3560] 619706 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.683 [api_impl.cc:2987] 619706 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.699 [runtime.cc:5239] 619706 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.712 [api_impl.cc:2992] 619706 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:44.755.730 [awatchdog_core.c:175](tid:619706) create watchdog for id : 65543, tid : 619706, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.755.746 [api_impl.cc:3025] 619706 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.755.761 [acl.cpp:115]619706 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.755.775 [acl.cpp:121]619706 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.755.788 [device.cpp:154]619706 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:44.755.826 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.222 msec. [WARNING] DEVICE(619447,fffed57aefa0,python3.9):2025-04-03-09:35:44.756.135 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 1 [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.900.182 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.900.183 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.900.207 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:44.900.215 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.220 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.229 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.253 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.258 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.900.269 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.900.273 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.900.285 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:44.900.292 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.301 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:44.900.308 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.068.754 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.068.789 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.068.825 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.068.842 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.068.875 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.068.944 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:45.256.370 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:45.256.408 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.549 msec. [WARNING] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:45.256.427 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:45.256.450 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.256.591 [device.cpp:143]619760 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.621 [api_impl.cc:2999] 619760 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.638 [device.cc:306] 619760 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.650 [runtime.cc:3393] 619760 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.666 [runtime.cc:3560] 619760 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.688 [api_impl.cc:2987] 619760 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.701 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.719 [api_impl.cc:2992] 619760 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:45.256.738 [awatchdog_core.c:175](tid:619760) create watchdog for id : 65543, tid : 619760, timeout : 300s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.256.754 [api_impl.cc:3025] 619760 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.256.765 [acl.cpp:115]619760 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.256.780 [acl.cpp:121]619760 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.256.792 [device.cpp:154]619760 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.308.355 [device.cpp:231]619760 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.385 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.573 [api_c_soc.cc:70] 619760 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.308.628 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.646 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.662 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.675 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.686 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.699 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=0. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.308.718 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=0.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.308.747 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.761 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.775 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.787 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.798 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.308.809 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=1. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.309.168 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=1.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.309.201 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.217 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.245 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.257 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.269 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.281 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=2. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.309.734 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=2.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.309.766 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.781 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.795 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.805 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.816 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.309.827 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=3. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.310.273 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=3.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.310.305 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.322 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.333 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.345 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.357 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.370 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=4. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.310.840 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=4.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.310.874 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.891 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.904 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.916 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.928 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.310.941 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=5. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.311.972 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=5.) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.312.007 [memory.cpp:559]619760 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.312.029 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.312.045 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.312.057 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.312.072 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.312.084 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=6. [INFO] DRV(619447,python3.9):2025-04-03-09:35:45.313.184 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=6.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.229 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.305 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.350 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.386 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.423 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.440 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.459 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.478 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.497 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.525 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.547 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.565 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.590 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.608 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.627 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.642 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.660 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.683 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.707 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.724 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.741 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.767 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.786 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.799 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.827 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.850 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.877 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.894 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.912 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.926 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.945 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.960 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.978 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.313.990 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.013 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.030 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.046 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.059 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.082 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.099 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.115 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.139 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.157 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.171 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.191 [runtime.cc:1664] 619760 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.206 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.230 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.245 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.279 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.308 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.329 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.344 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.358 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.374 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.394 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.423 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.436 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.449 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.465 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.481 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.508 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.520 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.534 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.550 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.564 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.592 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.606 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.619 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.633 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.648 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.674 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.689 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.702 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.717 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.732 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.769 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.782 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.794 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.808 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.824 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.847 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.860 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.314.876 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.314 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.369 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.412 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.426 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.439 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.455 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.476 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.514 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.528 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.541 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.555 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.577 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.610 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.625 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.637 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.653 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.674 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.714 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.728 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.754 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.770 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.786 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.824 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.838 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.850 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.866 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.887 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.929 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.945 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.958 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.973 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.315.988 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.031 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.044 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.057 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.073 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.095 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.135 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.148 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.187 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.207 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.225 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.266 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.282 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.297 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.325 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.347 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.386 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.399 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.412 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.427 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.443 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.483 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.497 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.511 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.526 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.547 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.587 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.601 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.615 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.628 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.643 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.681 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.696 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.710 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.725 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.748 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.783 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.798 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.811 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.827 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.843 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.890 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.904 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.919 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.935 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.316.961 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.000 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.014 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.028 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.042 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.061 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.103 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.116 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.130 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.143 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.158 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.200 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.212 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.226 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.243 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.258 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.296 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.310 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.323 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.341 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.356 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.398 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.413 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.435 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.449 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.470 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.508 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.520 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.533 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.547 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.563 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.602 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.614 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.626 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.317.675 [api_impl.cc:1990] 619760 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.318.000 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.318.033 [api_impl.cc:2664] 619760 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.395 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619419, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.424 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619423, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.445 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619427, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.464 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619431, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.479 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619435, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.497 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619439, num=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.512 [npu_driver.cc:3571] 619760 SetIpcMemPid: set ipc mem pid success, name=000973b7000000000000031a0700ffdc0007, pid[0]=619443, num=1 [2025-04-03 09:35:44.740877] [debug] [619447] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.740887] [debug] [619447] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.256916] [warn] [619760] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.256941] [debug] [619760] [lcal_sock_exchange.cpp:140] curRank: 7 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.256948] [debug] [619760] [lcal_sock_exchange.cpp:272] Client side 7 begin to connect [2025-04-03 09:35:45.257128] [debug] [619760] [lcal_sock_exchange.cpp:93] rank:7 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308330] [debug] [619760] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 7 rank :7 rankSize :8 [2025-04-03 09:35:45.308522] [debug] [619760] [lcal_comm.cpp:406] rank 7 devId: 7, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308532] [info] [619760] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308540] [info] [619760] [lcal_comm.cpp:242] rank 7/8 running devId:7 [2025-04-03 09:35:45.308594] [debug] [619760] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308621] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308738] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309191] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309756] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310296] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310866] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311998] [debug] [619760] [lcal_comm.cpp:340] 7 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313206] [debug] [619760] [lcal_comm.cpp:372] EnablePeerAccess succeed7 [2025-04-03 09:35:45.317642] [debug] [619760] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 7 [2025-04-03 09:35:45.317662] [debug] [619760] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.318021] [debug] [619760] [lcal_comm.cpp:455] peerMem[rank7], allocate finished. [2025-04-03 09:35:45.340290] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340308] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340317] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340325] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340333] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340339] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340345] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340350] [debug] [619760] [lcal_comm.cpp:472] rank : 7, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340357] [debug] [619760] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340526] [debug] [619760] [lcal_comm.cpp:557] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340835] [debug] [619760] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340847] [debug] [619760] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340855] [debug] [619760] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340862] [debug] [619760] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340879] [debug] [619760] [lcal_comm.cpp:515] rank[INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.340.938 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.342.025 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.342.045 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.342.888 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.342.904 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.343.564 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.343.581 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.343.949 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.343.965 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.358.003 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.358.023 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.359.129 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.359.146 [api_impl.cc:4286] 619760 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.040 [npu_driver.cc:3195] 619760 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.078 [api_impl.cc:1990] 619760 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.103 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.360.121 [memory.cpp:421]619760 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:45.360.278 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.360.312 [device.cpp:143]619760 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.335 [api_impl.cc:2999] 619760 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.348 [device.cc:306] 619760 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.363 [runtime.cc:3393] 619760 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.375 [runtime.cc:3560] 619760 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.389 [api_impl.cc:2987] 619760 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.403 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.419 [api_impl.cc:2992] 619760 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.433 [api_impl.cc:3025] 619760 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.360.445 [acl.cpp:115]619760 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.360.459 [acl.cpp:121]619760 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.360.484 [device.cpp:154]619760 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [WARNING] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:45.360.517 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.553 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.571 [adapter_rts.cc:198] [619760][hrtGetDeviceRefresh]deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.589 [externalinput.cc:2065] [619760]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.605 [externalinput.cc:490] [619760]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.622 [externalinput.cc:429] [619760]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.635 [externalinput.cc:543] [619760]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.650 [externalinput.cc:622] [619760]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.663 [externalinput.cc:714] [619760]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.677 [externalinput.cc:821] [619760]HCCL_IF_IP is not set [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.688 [externalinput.cc:877] [619760]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.701 [externalinput.cc:844] [619760]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.712 [externalinput.cc:728] [619760]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.725 [externalinput.cc:758] [619760]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.738 [externalinput.cc:806] [619760]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.751 [externalinput.cc:1508] [619760]HCCL_ALGO is not set [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.764 [externalinput.cc:1671] [619760][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.778 [externalinput.cc:462] [619760]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.790 [externalinput.cc:1519] [619760]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.803 [externalinput.cc:1554] [619760]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.360.895 [api_c_soc.cc:70] 619760 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.914 [externalinput.cc:1599] [619760]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.933 [externalinput.cc:1633] [619760]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.944 [externalinput.cc:1778] [619760]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.956 [externalinput.cc:1807] [619760]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.970 [externalinput.cc:515] [619760]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.360.982 [externalinput.cc:1841] [619760]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.006 [externalinput.cc:1929] [619760]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.019 [externalinput.cc:1949] [619760]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.032 [externalinput.cc:1969] [619760]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.055 [externalinput.cc:2005] [619760]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.072 [externalinput.cc:2036] [619760]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.087 [externalinput.cc:2209] [619760]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.100 [externalinput.cc:2178] [619760][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.114 [externalinput.cc:889] [619760]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.127 [externalinput.cc:914] [619760]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.139 [externalinput.cc:2240] [619760]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.150 [externalinput.cc:2308] [619760]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.162 [externalinput.cc:2339] [619760]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.176 [externalinput.cc:2296] [619760]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.189 [externalinput.cc:407] [619760]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:45.361.202 [externalinput.cc:672] [619760][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:45.361.223 [env_config.cc:127] [619760]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:45.361.239 [env_config.cc:131] [619760]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.254 [env_config.cc:302] [619760]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.267 [env_config.cc:302] [619760]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.277 [op_base.cc:1091] [619760]Entry-HcclCommInitRootInfoInner:ranks[8], rank[7], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.297 [op_base.cc:128] [619760][HcclGetDeviceId] deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.313 [op_base.cc:144] [619760][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.340 [rank_consistentcy_checker.cc:32] [619760][GetInstance] get deviceLogicId[7] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.361.365 [api_impl.cc:3100] 619760 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.361.379 [api_impl.cc:3111] 619760 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.361.394 [api_impl.cc:3130] 619760 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.588 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.607 [network_manager.cc:169] [619760]NetworkManager: devicePhyId[7], devicePhyId_[7] deviceLogicId_[7], hasBackup[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.361.635 [ra_host.c:1594]tid:619760,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.361.655 [adapter_hccp.cc:1450] [619760]set host socket whitelist status[0] success. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.361.689 [ra_host.c:205]tid:619760,ra_init(205) : Input parameters: phy_id[7], nic_position:[0] hdc_type:[6] [INFO] ROCE(619447,python3.9):2025-04-03-09:35:45.361.741 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.361.756 [ra_peer.c:776]tid:619760,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[7] start [INFO] ROCE(619447,python3.9):2025-04-03-09:35:45.361.811 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619447,python3.9):2025-04-03-09:35:45.361.866 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.361.892 [rs.c:327]tid:619760,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619447,python3.9):2025-04-03-09:35:45.362.023 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619447,python3.9):2025-04-03-09:35:45.362.048 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619447,python3.9):2025-04-03-09:35:45.362.062 [rs_ssl.c:1513]tid:619760,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.075 [rs_ssl.c:1529]tid:619760,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.102 [rs_epoll.c:692]tid:619760,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.254 [rs_epoll.c:487]tid:619766,rs_epoll_handle(487) : thread begin! thread_id:281469406670752, pid:619447, ppid:619411 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.291 [rs_epoll.c:713]tid:619760,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.304 [rs_epoll.c:415]tid:619766,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.302 [rs_epoll.c:623]tid:619768,rs_connect_handle(623) : thread begin! thread_id:281469398216608, pid:619447, ppid:619411 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.311 [rs.c:366]tid:619760,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.223000] ms [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.320 [rs_epoll.c:511]tid:619766,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.362.354 [rs_epoll.c:641]tid:619768,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.201 [rs.c:471]tid:619760,rs_init(471) : rs init success, chip_id[7] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.229 [ra_peer.c:813]tid:619760,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[7] succ [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.370.247 [adapter_hccp.cc:583] [619760]init ra success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.370.263 [network_manager.cc:222] [619760]NetworkManager nicDeploy[0] deviceLogicId_[7] devicePhyId_[7] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.283 [ra_host.c:1764]tid:619760,ra_get_ifnum(1764) : Input parameters: phy_id[7], nic_position:[0], is_all:[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.303 [ra_peer.c:859]tid:619760,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[7] start [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.344 [ra_peer.c:867]tid:619760,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[7] succ [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.370.358 [adapter_hccp.cc:1378] [619760][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.374 [ra_host.c:1814]tid:619760,ra_get_ifaddrs(1814) : Input parameters: phy_id[7], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.391 [ra_peer.c:874]tid:619760,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[7] start [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.411 [rs_socket.c:2743]tid:619760,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.423 [rs_socket.c:2743]tid:619760,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.434 [rs_socket.c:2743]tid:619760,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.447 [rs_socket.c:2749]tid:619760,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.460 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.471 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.483 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.493 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.503 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.514 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.527 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.540 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.553 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.563 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.574 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.585 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.597 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.611 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.624 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.637 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.651 [rs_socket.c:2749]tid:619760,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.663 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.677 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.691 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.701 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.723 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.738 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.752 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.762 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.772 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.784 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.795 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.806 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.817 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.829 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.843 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.856 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.868 [rs_socket.c:2749]tid:619760,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.879 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.891 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.902 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.913 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.924 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.935 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.946 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.959 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.972 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.982 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.370.992 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.003 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.014 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.024 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.038 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.049 [rs_socket.c:2751]tid:619760,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.060 [ra_peer.c:882]tid:619760,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[7] succ [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.087 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.103 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.118 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.131 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.145 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.157 [adapter_hccp.cc:1407] [619760][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.183 [sal.cc:365] [619760]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.198 [sal.cc:432] [619760]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.216 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:45.371.246 [ra_host.c:340]tid:619760,ra_socket_init_v1(340) : socket init:mode=0 phy_id=7 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.264 [adapter_hccp.cc:818] [619760]socket init v1 success, socketHandle[0xfffeac0faba0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.278 [network_manager.cc:1218] [619760]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.289 [network_manager.cc:1257] [619760]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffeac0faba0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.345 [adapter_hal.cc:298] [619760]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.368 [sal.cc:593] [619760]IsHostUseDevNic[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.379 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.391 [network_manager.cc:169] [619760]NetworkManager: devicePhyId[4294967295], devicePhyId_[7] deviceLogicId_[7], hasBackup[0] [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.440 [tsd_client.cpp:189][TsdCapabilityGet][tid:619760] TsdCapabilityGet Begin. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.456 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.469 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.481 [process_mode_manager.cpp:1042][CapabilityGet][tid:619760] [ProcessModeManager] enter into CapabilityGet process deviceId[7] type[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.494 [adapter_tdt.cc:66] [619760]Get TsdCapability success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.507 [network_manager.cc:175] [619760][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:45.371.525 [network_manager.cc:99] [619760][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.541 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.552 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.574 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619760] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.589 [process_mode_manager.cpp:386][InitTsdClient][tid:619760] [TsdClient] tsd client has been inited before [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.608 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619760] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619447,python3.9):2025-04-03-09:35:45.371.629 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619760] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.644 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619760] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.659 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619760] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.672 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619760] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.695 [version_verify.cpp:114][SpecialFeatureCheck][tid:619760] VersionVerify: previous type[48], supported [INFO] TDT(619447,python3.9):2025-04-03-09:35:45.371.742 [hdc_client.cpp:154][TsdRecvData][tid:619760] recv timeout is:150000 ms [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.412.177 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.412.180 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.412.204 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.412.213 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.222 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.232 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.238 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.248 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.412.256 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.412.263 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.412.269 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.412.275 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.295 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.412.306 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.166 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.175 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.174 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.174 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.178 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.572.178 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.289 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.306 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.324 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.326 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.341 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.349 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.357 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.357 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.364 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.372 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.379 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.385 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.397 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.401 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.408 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.415 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.431 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.437 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.446 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.441 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.442 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.458 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.465 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.470 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.471 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.483 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.488 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.496 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.500 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.506 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.511 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.517 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.572.530 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.535 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.542 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.572.555 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.924.168 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.924.175 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.924.188 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:45.924.198 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.201 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.231 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.237 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.247 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.924.251 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.924.258 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.924.263 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:45.924.272 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.277 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:45.924.287 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.284.727 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:46.284.751 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:46.284.765 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:46.284.780 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.168 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.176 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.177 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.176 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.197 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.404.201 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.291 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.302 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.308 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.316 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.333 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.336 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.341 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.348 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.353 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.358 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.364 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.368 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.371 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.377 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.384 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.392 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.397 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.401 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.402 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.408 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.414 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.427 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.443 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.428 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.438 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.451 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.455 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.463 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.475 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.483 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.488 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.404.509 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.521 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.528 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.534 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.404.540 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.432.178 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.432.203 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.432.222 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.432.239 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.432.252 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.432.269 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.432.287 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.436.171 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.436.194 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.436.208 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.436.224 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.436.239 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.436.251 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.436.266 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.948.172 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.948.172 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.948.192 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:46.948.199 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.204 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.227 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.235 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.241 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.948.246 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.948.254 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.948.258 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:46.948.268 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.274 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:46.948.282 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.167 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.171 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.173 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.176 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.183 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.236.191 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.272 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.291 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.295 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.303 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.308 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.317 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.325 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.324 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.330 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.354 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.356 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.360 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.369 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.374 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.381 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.388 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.393 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.404 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.411 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.414 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.423 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.425 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.429 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.434 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.447 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.456 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.461 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.467 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.472 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.480 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.491 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.498 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.236.503 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.515 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.535 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.236.549 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.460.170 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.460.173 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.460.193 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.460.201 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.206 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.219 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.226 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.233 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.460.239 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.460.250 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.460.254 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.460.264 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.273 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.460.279 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.731.545 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.731.548 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619760] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.731.586 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619760] [TsdClient] DeviceMsgProc recvMsg realDeviceId[7] msgType[49] localDevId[7] rspCode[0]heterogeneousSubPid[30993], tsdSupportLevel_[63] [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.731.607 [hdc_client.cpp:172][TsdRecvData][tid:619760] [deviceId=7] the Tsd recv data already [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.731.612 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.731.623 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619760] OpenSubProc success stype:0, pid:30993 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.731.640 [adapter_tdt.cc:39] [619760]Open TsdProcess success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.731.659 [network_manager.cc:118] [619760][TsdProcessOpen]hrtTsdProcessOpen success, subPid[30993], devicePhyId_[7], deviceLogicId_[7], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.731.694 [network_manager.cc:180] [619760][NetworkManager][Init]open tsd by process success, devicePhyId[7], deviceLogicId_[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.731.737 [ra_host.c:205]tid:619760,ra_init(205) : Input parameters: phy_id[7], nic_position:[1] hdc_type:[18] [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.731.755 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619447,python3.9):2025-04-03-09:35:47.731.772 [devdrv_hccl.c:103][ascend][curpid:619447,619760][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619447) [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.731.790 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.731.808 [ra_hdc.c:580]tid:619760,ra_hdc_init(580) : hdc init start! logic id is 7, phy id is 7, hdc_type is 18 [INFO] DRV(619447,python3.9):2025-04-03-09:35:47.732.040 [hdc_core.c:1982][ascend][curpid:619447,619760][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4267; pid=619447) [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.732.266 [ra_hdc.c:616]tid:619760,ra_hdc_init(616) : hdc init OK! phy_id[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.735.928 [adapter_hccp.cc:583] [619760]init ra success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.735.946 [network_manager.cc:222] [619760]NetworkManager nicDeploy[1] deviceLogicId_[7] devicePhyId_[7] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.735.973 [topoinfo_detect.cc:534] [619760]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.736.001 [ra_host.c:1764]tid:619760,ra_get_ifnum(1764) : Input parameters: phy_id[7], nic_position:[1], is_all:[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.384 [adapter_hccp.cc:1538] [619760][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.736.406 [ra_host.c:1814]tid:619760,ra_get_ifaddrs(1814) : Input parameters: phy_id[7], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.755 [adapter_hccp.cc:1568] [619760][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth7] ip[29.105.73.96%eth7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.777 [adapter_hccp.cc:1568] [619760][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth7] ip[fe80::2287:ecff:fe02:45fa%eth7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.792 [topoinfo_detect.cc:577] [619760]select AF_INET family as device socket family. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.809 [topoinfo_detect.cc:756] [619760]no backup device ip: use 0 as device ip. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.821 [topoinfo_detect.cc:294] [619760][HCCL_TRACE]SetupAgent rankNum[8], rank[7], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[7], phydevId[7], deviceIp[29.105.73.96] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.853 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.872 [hccl_socket.cc:516] [619760][Get][NicHandleInfo]phyId[7], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.887 [hccl_socket.cc:217] [619760][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281469273484192], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.736.904 [adapter_hccp.cc:1050] [619760]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.736.941 [ra_host.c:759]tid:619760,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[7], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.736.969 [rs_socket.c:1810]tid:619760,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.736.991 [rs_socket.c:1819]tid:619760,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619447,python3.9):2025-04-03-09:35:47.737.007 [rs_socket.c:132]tid:619760,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.023 [rs_socket.c:1880]tid:619760,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.220 [rs_socket.c:1577]tid:619768,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:59803, fd:53 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.252 [rs_drv_socket.c:59]tid:619768,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:53 begin [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.332 [rs_drv_socket.c:98]tid:619768,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:59803, server addr 192.168.0.44:60008, fd:53 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.354 [rs_socket.c:1427]tid:619768,rs_conn_cost_time(1427) : socket [53] connect success! cost [0.332000] ms [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.737.398 [rs_socket.c:1392]tid:619768,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.738.107 [topoinfo_exchange_agent.cc:212] [619760]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.738.169 [topoinfo_exchange_agent.cc:236] [619760]local rank[7] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.738.191 [topoinfo_exchange_agent.cc:43] [619760]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.738.208 [topoinfo_exchange_agent.cc:345] [619760]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.738.303 [topoinfo_exchange_agent.cc:77] [619760]topo exchange client send rank basic info success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.800.475 [api_impl.cc:2096] 619760 HostMalloc: size=2485 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.800.520 [api_error.cc:1260] 619760 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.800.821 [api_error.cc:1278] 619760 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.842 [topoinfo_exchange_agent.cc:80] [619760]topo exchange client get rank basic info success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.891 [topoinfo_detect.cc:303] [619760]topo detect completed. myrank[7], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.926 [config.cc:198] [619760]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.952 [config.cc:210] [619760]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.984 [config.cc:210] [619760]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.800.999 [config.cc:210] [619760]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.014 [config.cc:210] [619760]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.029 [config.cc:210] [619760]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.044 [config.cc:210] [619760]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.059 [config.cc:210] [619760]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.071 [config.cc:210] [619760]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.801.087 [adapter_hccp.cc:1067] [619760]ra socket batch close fdhandle[0xfffeac054bf0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.801.101 [ra_host.c:796]tid:619760,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[7], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.801.119 [rs_socket.c:1937]tid:619760,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:53, state:9 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.848.211 [adapter_hccp.cc:1099] [619760]ra socket batch close success,take time [47111]us [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:47.848.245 [hccl_socket.cc:232] [619760][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.848.266 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.292 [ra_host.c:404]tid:619760,ra_socket_deinit(404) : Input parameters: phy_id[7] family[2] local_ip[192.168.0.44] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.306 [rs.c:1629]tid:619760,rs_socket_deinit(1629) : rs socket deinit start, phy_id:7 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.320 [rs.c:1641]tid:619760,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.335 [rs.c:1653]tid:619760,rs_socket_deinit(1653) : socket deinit success, phy_id:7, local_ip:192.168.0.44 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.848.352 [network_manager.cc:1233] [619760]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.848.365 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.377 [ra_host.c:248]tid:619760,ra_deinit(248) : Input parameters: phy_id[7], nic_position:[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.388 [ra_peer.c:821]tid:619760,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[7] start [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.407 [rs.c:1783]tid:619760,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619447,python3.9):2025-04-03-09:35:47.848.411 [rs_epoll.c:523]tid:619766,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:51 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.848.432 [rs_epoll.c:544]tid:619766,rs_epoll_handle(544) : QUIT thread_id:281469406670752, pid:619447, read num:8 [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.947.902 [rs_epoll.c:679]tid:619768,rs_connect_handle(679) : QUIT thread_id:281469398216608, pid:619447 [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.948.705 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.948.729 [rs.c:1810]tid:619760,rs_deinit(1810) : rs_deinit chip_id[7] ok [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.948.742 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.948.759 [ra_peer.c:852]tid:619760,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[7] succ [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.948.771 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.948.798 [network_manager.cc:409] [619760]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.948.815 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.948.830 [ra_host.c:248]tid:619760,ra_deinit(248) : Input parameters: phy_id[7], nic_position:[1] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.948.843 [ra_hdc.c:651]tid:619760,ra_hdc_deinit(651) : hdc deinit start! phy_id[7] [INFO] DRV(619447,python3.9):2025-04-03-09:35:47.948.995 [hdc_client.c:417][ascend][curpid:619447,619760][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4267) [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.949.044 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:47.949.062 [ra_hdc.c:685]tid:619760,ra_hdc_deinit(685) : hdc deinit OK! phy_id[7] [INFO] ROCE(619447,python3.9):2025-04-03-09:35:47.949.076 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.102 [tsd_client.cpp:189][TsdCapabilityGet][tid:619760] TsdCapabilityGet Begin. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.118 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.134 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.147 [process_mode_manager.cpp:1042][CapabilityGet][tid:619760] [ProcessModeManager] enter into CapabilityGet process deviceId[7] type[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:47.949.160 [adapter_tdt.cc:66] [619760]Get TsdCapability success. deviceLogicId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.174 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.185 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.198 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619760] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.214 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619760] add close subproc:30993, proctype:0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.229 [version_verify.cpp:114][SpecialFeatureCheck][tid:619760] VersionVerify: previous type[60], supported [INFO] TDT(619447,python3.9):2025-04-03-09:35:47.949.259 [hdc_client.cpp:154][TsdRecvData][tid:619760] recv timeout is:150000 ms [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.972.172 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.972.176 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.972.201 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:47.972.212 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.238 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.243 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.251 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.255 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.972.262 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.972.265 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.972.273 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:47.972.280 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.288 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:47.972.294 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.168 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.171 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.172 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.182 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.182 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.068.184 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.281 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.301 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.308 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.317 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.322 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.333 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.338 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.340 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.360 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.367 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.373 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.376 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.384 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.392 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.400 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.405 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.406 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.417 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.427 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.431 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.436 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.438 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.450 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.455 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.465 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.469 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.472 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.478 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.487 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.493 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.500 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.068.515 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.523 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.556 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.560 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.068.570 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.202.122 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619760] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.202.146 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619760] [TsdClient] DeviceMsgProc recvMsg realDeviceId[7] msgType[61] localDevId[7] rspCode[0]heterogeneousSubPid[30993], tsdSupportLevel_[63] [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.202.165 [hdc_client.cpp:172][TsdRecvData][tid:619760] [deviceId=7] the Tsd recv data already [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.202.180 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619760] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.202.195 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619760] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.211 [adapter_tdt.cc:52] [619760]Open TsdProcess success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.222 [network_manager.cc:405] [619760][DeInit]finish CloseHccpProcess, phy_id[7], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.239 [network_manager.cc:409] [619760]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.251 [topoinfo_detect.cc:555] [619760]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.264 [topoinfo_detect.cc:321] [619760]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.281 [topoinfo_detect.cc:219] [619760]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.298 [rank_consistentcy_checker.cc:32] [619760][GetInstance] get deviceLogicId[7] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:48.202.314 [rank_consistentcy_checker.cc:144] [619760][Record][CannVersion] version information str is empty. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.325 [op_base.cc:2231] [619760]rank table is null, rankTableCrc is 0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.367 [op_base.cc:2219] [619760]params.id.internal [hccl-619447-0-1743644148-hccl_world_group] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.383 [op_base.cc:993] [619760]rootInfo[hccl-619447-0-1743644148-hccl_world_group], params.logiceDevice[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.396 [hccl_comm.cc:98] [619760]hcclComm init workmode [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.202.410 [hccl_comm.cc:954] [619760]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.246 [hccl_comm.cc:1365] [619760]RegistTaskAbortHandler begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.269 [task_abort_handler.cc:126] [619760]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.285 [api_impl.cc:4192] 619760 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.299 [runtime.cc:4698] 619760 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.350 [adapter_rts.cc:42] [619760]open so: libruntime.so [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.615 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.652 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.670 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.688 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.702 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.720 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.735 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.750 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.765 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.784 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.798 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.815 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.826 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.842 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.856 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.872 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.885 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.901 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.916 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.934 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.946 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.962 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.203.974 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.203.992 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.015 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.035 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.050 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.067 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.079 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.097 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.112 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.130 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.142 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.169 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.191 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.210 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.224 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.239 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.250 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.267 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.281 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.298 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.311 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.328 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.343 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.359 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.371 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.389 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.401 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.427 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.440 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.457 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.469 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.485 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.499 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.515 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.565 [hccl_communicator_attrs.cc:407] [619760]module[0] contains [8]devices [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.582 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.595 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.607 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.620 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.632 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.645 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.657 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.669 [hccl_communicator_attrs.cc:411] [619760]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.683 [hccl_communicator_attrs.cc:425] [619760]different module contains different numbers of cards:[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.697 [hccl_communicator_attrs.cc:426] [619760]different module contains different type of cards:[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.748 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.774 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.791 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.812 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.827 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.844 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.866 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.884 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.900 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.916 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.929 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.950 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.968 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.204.984 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.204.996 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.012 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.031 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.054 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.067 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.084 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.097 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.122 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.135 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.153 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.169 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.187 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.199 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.217 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.232 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.251 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.263 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.291 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.307 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.330 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.346 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.367 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.383 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.402 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.416 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.436 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.452 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.468 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.480 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.500 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.514 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.533 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.545 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.565 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.577 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.597 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.612 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.630 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.644 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.663 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.677 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.695 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.716 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.737 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.753 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.769 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.783 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.803 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.815 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.832 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.846 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.868 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.881 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.899 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.914 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.931 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.945 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.205.965 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.205.982 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.002 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.018 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.040 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.053 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.073 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.089 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.108 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.122 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.151 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.165 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.184 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.197 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.213 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.225 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.249 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.262 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.282 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.296 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.315 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.329 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.345 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.362 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.379 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.391 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.410 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.423 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.448 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.462 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.479 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.492 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.510 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.522 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.552 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.567 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.585 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.597 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.614 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.631 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.652 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.669 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.686 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.702 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.721 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.735 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.753 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.769 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.789 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.802 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.821 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.833 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.853 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.865 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.882 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.894 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.913 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.928 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.947 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.206.969 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.206.987 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.002 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.019 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.031 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.051 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.067 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.085 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.098 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.117 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.129 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.144 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.157 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.172 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.188 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.206 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.221 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.238 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.253 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.270 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.283 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.299 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.311 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.325 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.339 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.364 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.376 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.390 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.405 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.420 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.433 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.447 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.460 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.477 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.490 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.506 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.517 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.534 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.579 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.602 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.620 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.637 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.655 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.676 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.690 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.714 [hccl_communicator_attrs.cc:156] [619760][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.735 [hccl_communicator_attrs.cc:1021] [619760][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.752 [hccl_communicator.cc:1440] [619760][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.763 [hccl_communicator.cc:1450] [619760][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.778 [externalinput.cc:1445] [619760]InitParsesProtocol start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.796 [externalinput.cc:1462] [619760]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:48.207.812 [externalinput.cc:1411] [619760]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.835 [externalinput.cc:1413] [619760]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.848 [externalinput.cc:1467] [619760]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.887 [config.cc:557] [619760][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.903 [hccl_communicator.cc:668] [619760][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.928 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.948 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.207.963 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=0. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.207.982 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=0.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.207.999 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.015 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.032 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.044 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=1. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.059 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=1.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.074 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.087 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.099 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.112 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=2. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.129 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=2.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.168 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.190 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.208 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.223 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=3. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.237 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=3.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.250 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.264 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.279 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.291 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=4. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.305 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=4.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.317 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.328 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.342 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.354 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=5. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.368 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=5.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.382 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.393 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.407 [api_impl.cc:3520] 619760 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.208.420 [api_impl.cc:3168] 619760 EnableP2P: Enable P2P devIdDes=7, phyIdSrc=6. [INFO] DRV(619447,python3.9):2025-04-03-09:35:48.208.432 [devdrv_pcie.c:888][ascend][curpid:619447,619760][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=7; peer_phy_devid=6.) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.447 [p2p_mgmt.cc:74] [619760]enable p2p: local logic id:7, remote physic id:6. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.476 [adapter_hal.cc:298] [619760]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.499 [sal.cc:593] [619760]IsHostUseDevNic[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.551 [adapter_hal.cc:298] [619760]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.573 [sal.cc:593] [619760]IsHostUseDevNic[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.588 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.600 [network_manager.cc:169] [619760]NetworkManager: devicePhyId[4294967295], devicePhyId_[7] deviceLogicId_[7], hasBackup[0] [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.621 [tsd_client.cpp:189][TsdCapabilityGet][tid:619760] TsdCapabilityGet Begin. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.636 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.649 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.678 [process_mode_manager.cpp:1042][CapabilityGet][tid:619760] [ProcessModeManager] enter into CapabilityGet process deviceId[7] type[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.694 [adapter_tdt.cc:66] [619760]Get TsdCapability success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.709 [network_manager.cc:175] [619760][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:48.208.726 [network_manager.cc:99] [619760][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.746 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.762 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619760] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.773 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619760] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.787 [process_mode_manager.cpp:386][InitTsdClient][tid:619760] [TsdClient] tsd client has been inited before [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.804 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619760] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619447,python3.9):2025-04-03-09:35:48.208.824 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619760] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.840 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619760] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.853 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619760] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.870 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619760] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.208.916 [hdc_client.cpp:154][TsdRecvData][tid:619760] recv timeout is:150000 ms [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.480.180 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.480.212 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.480.231 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.480.246 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.480.263 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.480.293 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.480.309 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.484.171 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.484.195 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.484.212 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.484.227 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.484.243 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.484.257 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.484.272 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.171 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.169 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.173 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.175 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.178 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.900.181 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.290 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.307 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.320 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.326 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.332 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.340 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.345 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.354 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.356 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.360 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.377 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.391 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.394 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.396 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.407 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.414 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.419 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.427 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.435 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.438 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.445 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.447 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.454 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.461 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.467 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.470 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.473 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.484 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.493 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.498 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.512 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.519 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.900.526 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.532 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.538 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.900.567 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.940.723 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.940.755 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.940.771 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:48.940.784 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.996.167 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.996.167 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.996.192 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:48.996.199 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.209 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.215 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.222 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.230 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.996.236 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.996.242 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.996.248 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:48.996.256 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.262 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:48.996.271 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.508.170 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.508.174 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.508.196 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.508.205 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.209 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.222 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.245 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.251 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.508.258 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.508.263 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.508.271 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.508.276 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.286 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.508.295 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.728.171 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.728.277 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.728.298 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.728.314 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.728.327 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.728.338 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.728.350 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.732.169 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.732.170 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.732.170 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.732.172 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:49.732.185 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.269 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.286 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.306 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.320 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.326 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.335 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.342 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.341 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.346 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.357 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.364 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.370 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.374 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.375 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.381 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.398 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.403 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.405 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.411 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.419 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.426 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.432 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.441 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.448 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.454 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.458 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:49.732.472 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.479 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.483 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:49.732.509 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.020.167 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.020.170 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.020.193 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.020.201 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.209 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.215 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.223 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.231 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.020.236 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.020.245 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.020.251 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.020.258 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.267 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.020.271 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] TDT(619447,python3.9):2025-04-03-09:35:50.311.000 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619760] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.311.007 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:50.311.026 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619760] [TsdClient] DeviceMsgProc recvMsg realDeviceId[7] msgType[49] localDevId[7] rspCode[0]heterogeneousSubPid[31012], tsdSupportLevel_[63] [INFO] TDT(619447,python3.9):2025-04-03-09:35:50.311.052 [hdc_client.cpp:172][TsdRecvData][tid:619760] [deviceId=7] the Tsd recv data already [INFO] TDT(619447,python3.9):2025-04-03-09:35:50.311.067 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619760] OpenSubProc success stype:0, pid:31012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.311.075 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.311.083 [adapter_tdt.cc:39] [619760]Open TsdProcess success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.311.102 [network_manager.cc:118] [619760][TsdProcessOpen]hrtTsdProcessOpen success, subPid[31012], devicePhyId_[7], deviceLogicId_[7], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.311.131 [network_manager.cc:180] [619760][NetworkManager][Init]open tsd by process success, devicePhyId[7], deviceLogicId_[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:50.311.180 [ra_host.c:205]tid:619760,ra_init(205) : Input parameters: phy_id[7], nic_position:[1] hdc_type:[18] [INFO] ROCE(619447,python3.9):2025-04-03-09:35:50.311.246 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619447,python3.9):2025-04-03-09:35:50.311.268 [devdrv_hccl.c:103][ascend][curpid:619447,619760][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619447) [INFO] ROCE(619447,python3.9):2025-04-03-09:35:50.311.308 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:50.311.325 [ra_hdc.c:580]tid:619760,ra_hdc_init(580) : hdc init start! logic id is 7, phy id is 7, hdc_type is 18 [INFO] DRV(619447,python3.9):2025-04-03-09:35:50.311.460 [hdc_core.c:1982][ascend][curpid:619447,619760][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4274; pid=619447) [INFO] HCCP(619447,python3.9):2025-04-03-09:35:50.311.726 [ra_hdc.c:616]tid:619760,ra_hdc_init(616) : hdc init OK! phy_id[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.510 [adapter_hccp.cc:583] [619760]init ra success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.531 [network_manager.cc:222] [619760]NetworkManager nicDeploy[1] deviceLogicId_[7] devicePhyId_[7] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.550 [hccl_communicator_attrs.cc:1307] [619760][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.564 [hccl_communicator_attrs.cc:1275] [619760]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.583 [hccl_communicator.cc:2259] [619760][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.603 [adapter_hccp.cc:1650] [619760]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.779 [adapter_hccp.cc:2343] [619760]Get available Vnic info success, deviceIdType[0] deviceId[0x7] Vnic ip[192.168.2.192] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.823 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:50.315.843 [ra_host.c:280]tid:619760,ra_socket_init(280) : socket init:mode=1 phy_id=7 family=2 ip=192.168.2.192 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.859 [adapter_hccp.cc:806] [619760]socket init success, ip[3221399744] device id[7], socketHandle[0xfffeac0ffbb0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.876 [network_manager.cc:692] [619760]devicePhyId[7], ip[192.168.2.192] socket init OK [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.315.888 [network_manager.cc:432] [619760][Start][Vnic]trying to listen on ip[192.168.2.192] port[16666]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:50.315.903 [ra_host.c:875]tid:619760,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[7], local_ip[192.168.2.192], port[16666] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.221 [network_manager.cc:448] [619760][Start][Vnic]Listen on ip[192.168.2.192], port[16666] sucess, devPhyId[7], devLogicId[7], isAutoPort[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.245 [network_manager.cc:451] [619760]NetworkManager devicePhyId_[7] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.263 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.276 [hccl_socket.cc:106] [619760][HcclSocket][Listen] device[7] listens on ip[192.168.2.192] port[16666] success, socketType[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.291 [hccl_socket_manager.cc:56] [619760][Init][Server]ip[192.168.2.192] port[16666] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.322 [hccl_communicator.cc:1548] [619760][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.192] port[16666] success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.340 [hccl_communicator_attrs.cc:1307] [619760][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.353 [hccl_communicator_attrs.cc:1275] [619760]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.372 [hccl_communicator.cc:1565] [619760]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.387 [network_manager.cc:69] [619760]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.401 [hccl_communicator.cc:2028] [619760]profiling config information:options[], mode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.517 [adapter_trace.cc:21] [619760]Call TraceCreate, return value[0] [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.316.540 [atrace_api.c:61](tid:619760) AtraceCreateWithAttr start. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.316.627 [trace_rb_log.c:45](tid:619760) init ring buffer time finished, realTime 1743644150316621086, monotonicTime 4211952655369, freq 99996 kHz, minutesWest 480 minutes [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.316.647 [trace_rb_log.c:137](tid:619760) [HCCL_619760_7] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.316.667 [tracer_mgr_operate.c:66](tid:619760) create object HCCL_619760_7 success, exitSave(true), noLock(false). [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.316.681 [atrace_api.c:63](tid:619760) AtraceCreateWithAttr end. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.696 [adapter_trace.cc:70] [619760]Call TraceCreateWithAttr, return value[0xaaaafbe0bb08], Params: objName[HCCL_619760_7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.734 [rank_consistentcy_checker.cc:32] [619760][GetInstance] get deviceLogicId[7] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.316.766 [api_impl.cc:1486] 619760 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.316.793 [npu_driver.cc:4604] 619760 GetMaxStreamAndTask: deviceId=7, tsId=0, streamCount=1984. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.810 [task_exception_handler.cc:1212] [619760]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.830 [hccl_communicator_attrs.cc:1099] [619760][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.316.882 [api_impl.cc:4219] 619760 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.903 [command_handle.cc:25] [619760][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.316.918 [command_handle.cc:99] [619760][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.317.008 [profiler_manager_impl.cc:76] [619760][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.317.028 [hccl_communicator.cc:2193] [619760][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.317.044 [hccl_communicator.cc:803] [619760]Will use ffts mode. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.317.070 [api_impl.cc:3138] 619760 GetDeviceIndexByPhyId: get Index by PhyId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.317.090 [api_impl.cc:3149] 619760 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.317.114 [api_impl.cc:3157] 619760 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.317.126 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.318.611 [api_impl.cc:3476] 619760 GetDeviceInfo: get device info, deviceId=7 [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:50.318.658 [tbe_vector_reduce.cc:508] [619760]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.305 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.465 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.319.486 [dispatcher.cc:157] [619760][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.506 [api_impl.cc:3138] 619760 GetDeviceIndexByPhyId: get Index by PhyId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.523 [api_impl.cc:3149] 619760 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.535 [api_impl.cc:3157] 619760 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.548 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.319.576 [api_impl.cc:1990] 619760 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.320.427 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.320.500 [api_impl.cc:1990] 619760 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.320.526 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.320.559 [api_impl.cc:1990] 619760 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.320.580 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.613 [opexecounter.cc:71] [619760]alloc counter mem resource. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.653 [hccl_communicator_attrs.cc:1368] [619760][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.706 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.730 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.753 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.769 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.784 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.797 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.813 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.852 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.868 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.883 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.898 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.911 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.925 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.939 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.952 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.966 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.983 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.320.996 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.009 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.024 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.037 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.051 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.066 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.079 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.093 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.108 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.122 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.137 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.152 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.163 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.187 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.201 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.215 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.231 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.246 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.258 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.271 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.286 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.298 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.310 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.327 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.338 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.354 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.370 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.381 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.397 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.411 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.424 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.439 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.452 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.463 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.477 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.501 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.515 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.530 [alg_configurator.cc:68] [619760][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.544 [alg_configurator.cc:241] [619760][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.557 [alg_configurator.cc:134] [619760]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.580 [alg_configurator.cc:449] [619760][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.596 [alg_configurator.cc:42] [619760]alg configurator init success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.628 [topo_info_extractor.cc:66] [619760]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[7], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.652 [topo_info_extractor.cc:155] [619760][SetRankInfo]rankData_: userRank[7], devicePhyId[7], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.678 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.708 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.742 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.769 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.789 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.813 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.828 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.851 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.884 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.901 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.915 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.935 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.321.950 [api_error.cc:3478] 619760 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.970 [adapter_rts.cc:940] [619760][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.321.989 [topo_info_extractor.cc:348] [619760]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.144 [topo_info_extractor.cc:888] [619760]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[7], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.186 [topo_info_extractor.cc:585] [619760][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.209 [topo_info_extractor.cc:646] [619760][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.225 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.244 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.263 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.279 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.297 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.312 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.325 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.340 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/1; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.353 [topo_info_extractor.cc:689] [619760]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[7], userRankSize[8], plane size[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.366 [topo_info_extractor.cc:697] [619760][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.379 [topo_info_extractor.cc:734] [619760]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 7/7/0/; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.391 [topo_info_extractor.cc:739] [619760]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[7], userRankSize[8], plane size[1] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.406 [topo_info_extractor.cc:585] [619760][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.421 [topo_info_extractor.cc:646] [619760][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.435 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.462 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.494 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.512 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.526 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.543 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.559 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.573 [topo_info_extractor.cc:685] [619760]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/1; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.588 [topo_info_extractor.cc:689] [619760]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[7], userRankSize[8], plane size[8] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.625 [topo_info_extractor.cc:790] [619760]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId7/7; ] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.643 [topo_info_extractor.cc:519] [619760]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.322.775 [api_impl.cc:1990] 619760 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.322.805 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.322.855 [hccl_impl.cc:59] [619760]start hccl resources build:get deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.323.003 [hccl_impl.cc:177] [619760]hcclImpl init success. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:50.323.034 [hccl_aiv.cc:279] [619760][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.326 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.393 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.506 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.527 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.549 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.572 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.598 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.703 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.721 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.754 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.774 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.799 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.906 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.924 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.939 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.957 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.325.980 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.076 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.094 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.109 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.129 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.149 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.247 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.266 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.282 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.299 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.319 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.417 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.434 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.449 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.467 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.485 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.579 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.597 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.614 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.631 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.662 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.757 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.775 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.792 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.807 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.826 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.922 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.940 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.957 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.973 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.326.993 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.083 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.101 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.117 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.132 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.153 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.243 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.260 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.275 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.289 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.308 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.399 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.417 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.432 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.446 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.467 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.578 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.596 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.610 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.626 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.644 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.736 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.754 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.770 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.783 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.802 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.892 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.909 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.924 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.939 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.327.957 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.048 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.066 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.083 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.098 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.116 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.219 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.241 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.257 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.272 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.290 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.382 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.401 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.426 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.442 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.465 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.556 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.573 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.589 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.604 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.623 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.712 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.731 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.748 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.763 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.781 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.881 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.900 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.915 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.929 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.328.948 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.044 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.061 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.076 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.090 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.108 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.200 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.218 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.234 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.258 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.281 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.374 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.393 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.409 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.424 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.442 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.533 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.550 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.566 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.580 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.598 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.686 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.705 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.721 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.734 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.752 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.844 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.862 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.877 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.893 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.329.912 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.004 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.021 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.038 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.053 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.084 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.178 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.197 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.212 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.226 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.243 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.334 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.352 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.367 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.381 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.398 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.492 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.511 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.526 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.540 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.558 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.649 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.667 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.682 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.696 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.714 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.806 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.824 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.839 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.853 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.871 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.962 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.330.988 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.004 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.019 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.038 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.131 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.149 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.163 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.177 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.196 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.288 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.306 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.321 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.336 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.354 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.447 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.465 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.480 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.494 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.512 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.604 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.622 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.638 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.653 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.671 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.767 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.785 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.810 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.824 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.844 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.939 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.956 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.971 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.331.985 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.004 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.097 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.113 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.128 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.144 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.169 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.266 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.288 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.305 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.320 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.339 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.431 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.448 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.464 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.478 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.496 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.591 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.610 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.626 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.640 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.675 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.768 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.786 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.801 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.815 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.832 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.921 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.938 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.955 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.969 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.332.987 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.081 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.098 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.112 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.126 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.146 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.238 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.256 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.273 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.289 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.308 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.401 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.421 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.435 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.449 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.468 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.570 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.588 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.604 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.617 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.637 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.727 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.747 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.763 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.777 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.796 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.891 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.910 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.925 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.941 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.333.962 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.055 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.073 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.091 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.105 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.126 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.217 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.234 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.251 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.266 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.287 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.377 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.405 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.420 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.435 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.454 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.544 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.562 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.577 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.591 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.615 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.707 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.725 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.741 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.754 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.772 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.872 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.890 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.904 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.919 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.334.942 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.033 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.054 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.070 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.085 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.106 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.201 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.220 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.235 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.259 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.280 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.377 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.395 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.412 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.432 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.452 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.548 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.566 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.582 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.598 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.619 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.707 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.725 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.740 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.758 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.778 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.864 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.882 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.898 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.915 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.335.936 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.024 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.042 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.057 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.073 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.104 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.213 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.238 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.255 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.271 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.292 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.383 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.402 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.419 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.433 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.455 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.551 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.569 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.586 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.602 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.622 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.718 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.738 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.756 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.772 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.793 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.884 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.903 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.919 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.935 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.336.955 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.045 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.073 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.090 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.106 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.128 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.223 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.241 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.256 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.271 [runtime.cc:1673] 619760 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.292 [elf.cc:566] 619760 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.389 [runtime.cc:1688] 619760 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.406 [runtime.cc:1953] 619760 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.422 [runtime.cc:1897] 619760 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619447,python3.9):2025-04-03-09:35:50.337.446 [devmm_svm.c:1972][ascend][curpid:619447,619760][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.337.463 [hdc.cc:83] [619760][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.479 [api_impl.cc:1990] 619760 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.337.506 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.009 [api_impl.cc:1990] 619760 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.032 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.051 [api_impl.cc:2664] 619760 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619447,python3.9):2025-04-03-09:35:50.338.217 [devmm_svm.c:1972][ascend][curpid:619447,619760][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.237 [hdc.cc:83] [619760][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.251 [api_impl.cc:1990] 619760 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.267 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.287 [api_impl.cc:2096] 619760 HostMalloc: size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.306 [api_error.cc:1260] 619760 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.482 [api_impl.cc:1990] 619760 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.567 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.590 [api_impl.cc:2664] 619760 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.730 [api_impl.cc:1990] 619760 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.754 [api_error.cc:1218] 619760 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.773 [api_impl.cc:2664] 619760 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.802 [peterson_lock.cc:69] [619760][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.817 [peterson_lock.cc:47] [619760][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.848 [hccl_comm.cc:129] [619760]hcclCommInitInfo:commId[hccl-619447-0-1743644148-hccl_world_group], rank[7], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[7], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.875 [hccl_comm.cc:175] [619760][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.893 [topo_matcher.cc:528] [619760][SetAivMode]AivMode is set to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.906 [topo_matcher.cc:540] [619760][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.925 [hcom_plugin.cc:349] [619760][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.338.959 [runtime.cc:5239] 619760 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.971 [adapter_rts.cc:198] [619760][hrtGetDeviceRefresh]deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.338.991 [hcom_common.cc:85] [619760][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.008 [op_base.cc:128] [619760][HcclGetDeviceId] deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.022 [hccl_comm.cc:1296] [619760]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffed4f9ccb8] topoSize[2] [INFO] GE(619447,python3.9):2025-04-03-09:35:50.339.046 [hcom_topo_info.cc:24]619760 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.059 [hcom_plugin.cc:371] [619760][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.080 [topoinfo_detect.cc:555] [619760]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.094 [topoinfo_detect.cc:321] [619760]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.107 [topoinfo_detect.cc:219] [619760]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.127 [op_base.cc:1050] [619760][InitCommRootInfo]Init success, rankNum[8], rank[7], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.146 [profiler_base.cc:342] [619760]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.173 [op_base.cc:1107] [619760][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4978625]us, rankNum[8], rank[7] [WARNING] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.339.242 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.339.277 [op_base.cc:853] [619760]HcclGetCommName input handle=0xfffeac022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.339.308 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 7 [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.339.330 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 7, group name: hccl_world_group [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.339.438 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.339.462 [device.cpp:183]619760 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 7. [INFO] DEVICE(619447,fffeb37eefa0,python3.9):2025-04-03-09:35:50.339.470 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 7, group name: hccl_world_group [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.339.487 [api_impl.cc:3419] 619760 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.339.502 [runtime.cc:3444] 619760 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.339.516 [api_impl.cc:3449] 619760 DeviceReset: Succ, userDeviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.339.528 [device.cpp:189]619760 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 7. [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.339.550 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:50.339.637 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5083.12 msec. [WARNING] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:50.339.673 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.339.748 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5584.62 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.339.787 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5584.68 msec. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.339.809 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.339.837 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9742.17 msec. [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.339.875 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.540 [device.cpp:143]619447 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.582 [api_impl.cc:2999] 619447 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.604 [device.cc:306] 619447 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.622 [runtime.cc:3393] 619447 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.642 [runtime.cc:3560] 619447 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.659 [api_impl.cc:2987] 619447 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.673 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.697 [api_impl.cc:2992] 619447 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.718 [api_impl.cc:3025] 619447 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.733 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.748 [acl.cpp:121]619447 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.764 [device.cpp:154]619447 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.355.819 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.846 [context.cpp:174]619447 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.355.872 [api_impl.cc:6913] 619447 CtxSetSysParamOpt: curCtx = 0xaaab0ca695c0, configOpt=0, configVal=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.355.912 [context.cpp:158]619447 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.356.540 [op_base.cc:1214] [619447][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.356.570 [op_base.cc:128] [619447][HcclGetDeviceId] deviceLogicId[7] [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.356.617 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.356.639 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.356.664 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.356.712 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.357.357 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.357.952 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.046 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.232 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 197.14 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.276 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.322 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 44.54 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.349 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.372 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 24.24 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.393 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.461 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 65.31 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.506 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.562 msec. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.618 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.679 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.13 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.762 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 52.52 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.808 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.187 msec. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.841 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.881 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.933 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 48.9 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.957 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.358.990 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 31.07 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.012 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.032 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 19 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.061 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.098 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 34.64 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.143 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.298 msec. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.176 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.289 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.363 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 70.52 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.427 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 34.52 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.474 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 21.77 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.537 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 36.65 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.600 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 34.41 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.743 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 112.901 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.774 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 3.12 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.798 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.882 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 60.74 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.938 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 27.58 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.359.961 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.030 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 48.13 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.578 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 515.421 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.609 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.689 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 39.28 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.743 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.562 msec. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.767 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.360.824 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 2.204 msec. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.270 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 64.04 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.301 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.391 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 85.65 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.416 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.479 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 59.72 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.505 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.541 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 34.46 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.585 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.608 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 40.83 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.657 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 24.96 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.707 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.731 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 49.781 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.753 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.773 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 19.48 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.790 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.823 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 31.19 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.861 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.880 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 35.65 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.900 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.919 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 18.73 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.938 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.954 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 15.74 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.361.969 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.011 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 39.39 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.048 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 35.02 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.086 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.106 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 18.75 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.123 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.141 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 16.87 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.159 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.176 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 15.7 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.210 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.230 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 36.1 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.264 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.294 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 45.81 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.314 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.380 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 61.571 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.459 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.482 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 73.6 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.502 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.519 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 17.39 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.538 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.555 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 15.86 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.575 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 16.02 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.609 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.625 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 16.25 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.659 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.678 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 33.27 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.719 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.737 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 37.53 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.757 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.778 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 20.74 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.796 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.818 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 21.31 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.362.925 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 73.73 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 180.611 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.162 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.222 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 35.07 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.242 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.315 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 53.77 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.339 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 26.02 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.408 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.461 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 27.61 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.482 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.525 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 22.16 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.542 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.621 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 58.51 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.645 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.714 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 46.95 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.735 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.821 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 66.52 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.857 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.933 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 54.45 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.957 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.363.981 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 139.23 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.146 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.187 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.324 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 135.65 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.349 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.443 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 71.101 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.470 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.554 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 61.78 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.579 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.599 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.616 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.637 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.657 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.674 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.690 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.722 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.740 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.756 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.792 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.808 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.826 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.364.842 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.107 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 241.071 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.155 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.173 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.190 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.207 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.306 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 77.06 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.332 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.353 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.389 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.472 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 58.51 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.533 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.652 msec. [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.365.890 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.007 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.291 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 218.22 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.422 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 34.63 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.539 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 81.951 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.586 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 16.63 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.644 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 30.71 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.746 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 74.21 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.804 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 30.76 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.859 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 27.14 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.928 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 42.49 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.366.963 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 8.38 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.015 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.655 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.077 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.069 msec. [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.126 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.512 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.556 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.445 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.591 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.613 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.730 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 44.2 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.823 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 63.53 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.896 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 44.34 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.367.997 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 72.56 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.022 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.042 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.103 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 39.82 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 15.47 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.179 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 1.09 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.232 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.593 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.289 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.65 msec. [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.333 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.447 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.474 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 1.35 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 35.201 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.562 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 5.55 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.600 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.75 us [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.639 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.722 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 96.57 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.749 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 1.67 us [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.774 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.355 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.829 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.511 msec. [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.878 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.368.963 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.094 msec. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.369.487 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.369.518 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.369.539 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.369.587 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.369.856 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.369.909 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.012 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.126 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.123 msec. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.215 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.275 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.303 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.325 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.345 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.385 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.445 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.370.705 [context.cpp:72]619794 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.370.739 [api_impl.cc:3717] 619794 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.370.763 [awatchdog_core.c:175](tid:619794) create watchdog for id : 65543, tid : 619794, timeout : 300s [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.370.783 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.370.782 [context.cpp:80]619794 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.370.846 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.370.896 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.370.923 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61027123200, align_size:61027123200 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.372.315 [api_impl.cc:7115] 619794 ReserveMemAddress: device malloc Succ, size=61027123200(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5800000 [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.379.600 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.638 [api_impl.cc:7129] 619794 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.755 [api_impl.cc:7145] 619794 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.379.824 [device.cpp:143]619794 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.848 [api_impl.cc:2999] 619794 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.863 [device.cc:306] 619794 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.878 [runtime.cc:3393] 619794 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.893 [runtime.cc:3560] 619794 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.907 [api_impl.cc:2987] 619794 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.924 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.941 [api_impl.cc:2992] 619794 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.379.957 [api_impl.cc:3025] 619794 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.379.986 [acl.cpp:115]619794 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.379.999 [acl.cpp:121]619794 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.011 [device.cpp:154]619794 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.380.041 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.060 [context.cpp:174]619794 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.082 [api_impl.cc:6913] 619794 CtxSetSysParamOpt: curCtx = 0xaaab0ca695c0, configOpt=0, configVal=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.098 [context.cpp:158]619794 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.380.149 [op_base.cc:1214] [619794][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.197 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.380.215 [adapter_rts.cc:234] [619794][hrtGetDevice]deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:50.380.230 [op_base.cc:128] [619794][HcclGetDeviceId] deviceLogicId[7] [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.265 [context.cpp:72]619794 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.282 [api_impl.cc:3717] 619794 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.326 [context.cpp:80]619794 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.347 [api_error.cc:1807] 619794 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.369 [memory_task.cc:342] 619794 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.385 [memory_task.cc:473] 619794 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.397 [memory_task.cc:518] 619794 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.415 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.438 [memory_task.cc:1007] 619794 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaab1858f090, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.452 [memory_task.cc:1166] 619794 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.471 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.488 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.513 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.532 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.568 [callback.cpp:97]619794 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.595 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.616 [task_info.cc:521] 619794 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.634 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.650 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.669 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.687 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.699 [callback.cpp:109]619794 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.730 [npu_driver.cc:3844] 619600 CqReportGet: cq report get start, device_id=7, tsId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.757 [npu_driver.cc:3851] 619600 CqReportGet: halCqReportGet count=1 [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.380.772 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.771 [api_impl.cc:5087] 619600 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.380.806 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.803 [callback.cpp:138]619600 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.380.827 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61027123200, align_size:61027123200 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.380.826 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.380.844 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.382.165 [api_impl.cc:7115] 619794 ReserveMemAddress: device malloc Succ, size=61027123200(bytes) start ptr=0x12d100000000, end ptr=0x12df35800000 [INFO] PRE_ACT(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.389.931 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.389.965 [api_impl.cc:7129] 619794 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.073 [api_impl.cc:7145] 619794 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.390.174 [device.cpp:231]619794 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.197 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.390.236 [acl.cpp:802]619794 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.348 [npu_driver.cc:404] 619794 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.385 [stream.cc:2246] 619794 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.422 [stream_sqcq_manage.cc:147] 619794 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.440 [npu_driver.cc:882] 619794 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.777 [npu_driver.cc:6125] 619794 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.830 [npu_driver.cc:6137] 619794 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=344, streamId=6, logicCq=344, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.855 [npu_driver.cc:6189] 619794 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=6, logicCq=344. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.872 [npu_driver.cc:5916] 619794 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f4045000, addr=dffff4045000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.887 [stream.cc:653] 619794 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.931 [stream.cc:276] 619794 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.390.948 [stream.cc:688] 619794 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=7, streamResId=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.390.965 [acl_resource_manager.cpp:814]619794 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.390.979 [acl_resource_manager.cpp:816]619794 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.129 [engine.cc:188] 619794 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.153 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.170 [task_info.cc:409] 619794 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.183 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.198 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.217 [npu_driver.cc:751] 619794 SqTaskSend: device_id=7, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.233 [stars_engine.cc:893] 619794 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.250 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.264 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.280 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.295 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.311 [logger.cc:152] 619794 TaskFinished: device_id=7, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.348 [stars_engine.cc:437] 619794 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.366 [stream.cc:2886] 619794 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.385 [stream.cc:852] 619794 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.401 [stream.cc:938] 619794 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.416 [stream.cc:1507] 619794 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.440 [stream_sqcq_manage.cc:451] 619794 FreeLogicCq: Return(0), threadIdentifier(281469959925664), devId(7), tsId(0), cqId(344), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.459 [npu_driver.cc:6163] 619794 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=6, logicCq=344. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.489 [npu_driver.cc:1180] 619794 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=344, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.506 [stream.cc:237] 619794 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.520 [stream_sqcq_manage.cc:216] 619794 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.391.534 [npu_driver.cc:966] 619794 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.392.071 [npu_driver.cc:974] 619794 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.392.093 [stream_sqcq_manage.cc:228] 619794 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.392.110 [npu_driver.cc:432] 619794 StreamIdFree: stream id free begin, device_id=7, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.392.127 [npu_driver.cc:441] 619794 StreamIdFree: Stream id free success, stream_id=6, device_id=7, tsId=0, streamFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.392.142 [device_msg_handler.cc:110] 619794 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.393.940 [device_msg_handler.cc:127] 619794 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.394.053 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.394.089 [npu_driver.cc:500] 619794 GetFaultEvent: get fault event, device_id=7, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.394.170 [acl.cpp:782]619794 GetFaultEventInfo: can not get fault event of device 7, runtime errorCode is 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:50.394.200 [error_manager.cc:989]619794 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.394.521 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.394.570 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.394.593 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.394.610 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.394.840 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.394.989 [context.cpp:72]619794 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.014 [api_impl.cc:3717] 619794 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.032 [context.cpp:80]619794 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.053 [api_error.cc:1807] 619794 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.076 [memory_task.cc:342] 619794 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.395.082 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.090 [memory_task.cc:473] 619794 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.108 [memory_task.cc:518] 619794 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.127 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.143 [memory_task.cc:1007] 619794 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaab1858fab0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.161 [memory_task.cc:1166] 619794 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.175 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.189 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.209 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.227 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.248 [callback.cpp:97]619794 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.268 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.286 [task_info.cc:521] 619794 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.300 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.314 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.346 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.362 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.373 [callback.cpp:109]619794 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.395 [npu_driver.cc:3844] 619600 CqReportGet: cq report get start, device_id=7, tsId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.412 [npu_driver.cc:3851] 619600 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.423 [api_impl.cc:5087] 619600 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.440 [callback.cpp:138]619600 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.450 [device.cpp:231]619794 aclrtGetDevice: start to execute aclrtGetDevice [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.458 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.467 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.475 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.395.482 [acl.cpp:802]619794 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.515 [npu_driver.cc:404] 619794 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.544 [stream.cc:2246] 619794 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.585 [stream_sqcq_manage.cc:147] 619794 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.603 [npu_driver.cc:882] 619794 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.851 [npu_driver.cc:6125] 619794 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.895 [npu_driver.cc:6137] 619794 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=345, streamId=7, logicCq=345, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.918 [npu_driver.cc:6189] 619794 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=7, logicCq=345. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.936 [npu_driver.cc:5916] 619794 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f4045000, addr=dffff4045000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.954 [stream.cc:653] 619794 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.395.993 [stream.cc:276] 619794 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.011 [stream.cc:688] 619794 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=7, streamResId=65534 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.396.028 [acl_resource_manager.cpp:814]619794 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.396.040 [acl_resource_manager.cpp:816]619794 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.083 [engine.cc:188] 619794 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.115 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.134 [task_info.cc:409] 619794 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.149 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.178 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.195 [npu_driver.cc:751] 619794 SqTaskSend: device_id=7, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.209 [stars_engine.cc:893] 619794 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.223 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.239 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.254 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.267 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.279 [logger.cc:152] 619794 TaskFinished: device_id=7, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.298 [stars_engine.cc:437] 619794 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.316 [stream.cc:2886] 619794 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.330 [stream.cc:852] 619794 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.344 [stream.cc:938] 619794 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.357 [stream.cc:1507] 619794 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.404 [stream_sqcq_manage.cc:451] 619794 FreeLogicCq: Return(0), threadIdentifier(281469959925664), devId(7), tsId(0), cqId(345), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.427 [npu_driver.cc:6163] 619794 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=7, logicCq=345. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.457 [npu_driver.cc:1180] 619794 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=345, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.475 [stream.cc:237] 619794 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.491 [stream_sqcq_manage.cc:216] 619794 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.503 [npu_driver.cc:966] 619794 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.973 [npu_driver.cc:974] 619794 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.396.992 [stream_sqcq_manage.cc:228] 619794 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.006 [npu_driver.cc:432] 619794 StreamIdFree: stream id free begin, device_id=7, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.036 [npu_driver.cc:441] 619794 StreamIdFree: Stream id free success, stream_id=7, device_id=7, tsId=0, streamFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.053 [device_msg_handler.cc:110] 619794 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.399 [device_msg_handler.cc:127] 619794 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.434 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.397.457 [npu_driver.cc:500] 619794 GetFaultEvent: get fault event, device_id=7, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.397.515 [acl.cpp:782]619794 GetFaultEventInfo: can not get fault event of device 7, runtime errorCode is 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:50.397.534 [error_manager.cc:989]619794 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.397.828 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.397.862 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.397.885 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.397.904 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.397.981 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.061 [context.cpp:72]619794 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.084 [api_impl.cc:3717] 619794 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.103 [context.cpp:80]619794 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.129 [api_error.cc:1807] 619794 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.149 [memory_task.cc:342] 619794 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.162 [memory_task.cc:473] 619794 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.179 [memory_task.cc:518] 619794 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.195 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.210 [memory_task.cc:1007] 619794 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaab18590520, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.227 [memory_task.cc:1166] 619794 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.242 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.269 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.287 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.302 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.319 [callback.cpp:97]619794 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.340 [task_res.cc:380] 619794 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.357 [task_info.cc:521] 619794 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.373 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.389 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.409 [task_submit.cc:573] 619794 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.425 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.440 [callback.cpp:109]619794 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.444 [npu_driver.cc:3844] 619600 CqReportGet: cq report get start, device_id=7, tsId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.460 [npu_driver.cc:3851] 619600 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.473 [api_impl.cc:5087] 619600 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.489 [callback.cpp:138]619600 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.502 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.513 [device.cpp:231]619794 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.516 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.532 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.398.548 [acl.cpp:802]619794 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.577 [npu_driver.cc:404] 619794 StreamIdAlloc: stream id alloc begin, device_id=7, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.608 [stream.cc:2246] 619794 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.648 [stream_sqcq_manage.cc:147] 619794 Alloc: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.666 [npu_driver.cc:882] 619794 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.898 [npu_driver.cc:6125] 619794 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.951 [npu_driver.cc:6137] 619794 LogicCqAllocateV2: success: deviceId=7, tsId=0, cqId=346, streamId=8, logicCq=346, cqeDepth=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.975 [npu_driver.cc:6189] 619794 StreamBindLogicCq: success: deviceId=7, tsId=0, streamId=8, logicCq=346. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.398.992 [npu_driver.cc:5916] 619794 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=f4045000, addr=dffff4045000size=4096 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.009 [stream.cc:653] 619794 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.048 [stream.cc:276] 619794 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.067 [stream.cc:688] 619794 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=7, streamResId=65534 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.399.086 [acl_resource_manager.cpp:814]619794 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.399.100 [acl_resource_manager.cpp:816]619794 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.143 [engine.cc:188] 619794 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.163 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.182 [task_info.cc:409] 619794 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.195 [stream.cc:1400] 619794 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.210 [stream.cc:2595] 619794 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.227 [npu_driver.cc:751] 619794 SqTaskSend: device_id=7, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.240 [stars_engine.cc:893] 619794 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.254 [stream.cc:1494] 619794 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.268 [npu_driver.cc:5547] 619794 GetSqHead: deviceId=7, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.284 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.299 [task.cc:185] 619794 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.312 [logger.cc:152] 619794 TaskFinished: device_id=7, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.329 [stars_engine.cc:437] 619794 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.343 [stream.cc:2886] 619794 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.357 [stream.cc:852] 619794 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.370 [stream.cc:938] 619794 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.386 [stream.cc:1507] 619794 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.422 [stream_sqcq_manage.cc:451] 619794 FreeLogicCq: Return(0), threadIdentifier(281469959925664), devId(7), tsId(0), cqId(346), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.445 [npu_driver.cc:6163] 619794 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=8, logicCq=346. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.479 [npu_driver.cc:1180] 619794 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=346, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.497 [stream.cc:237] 619794 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.511 [stream_sqcq_manage.cc:216] 619794 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.399.523 [npu_driver.cc:966] 619794 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.086 [npu_driver.cc:974] 619794 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.107 [stream_sqcq_manage.cc:228] 619794 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.127 [npu_driver.cc:432] 619794 StreamIdFree: stream id free begin, device_id=7, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.145 [npu_driver.cc:441] 619794 StreamIdFree: Stream id free success, stream_id=8, device_id=7, tsId=0, streamFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.171 [device_msg_handler.cc:110] 619794 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.521 [device_msg_handler.cc:127] 619794 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.557 [runtime.cc:5239] 619794 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.400.582 [npu_driver.cc:500] 619794 GetFaultEvent: get fault event, device_id=7, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.400.641 [acl.cpp:782]619794 GetFaultEventInfo: can not get fault event of device 7, runtime errorCode is 0 [INFO] GE(619447,python3.9):2025-04-03-09:35:50.400.661 [error_manager.cc:989]619794 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.400.958 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.400.994 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.401.016 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.401.036 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619447,fffed4f9efa0,python3.9):2025-04-03-09:35:50.401.107 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:50.479.209 [awatchdog_monitor.c:160](tid:619479) thread 619760 not exist [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.532.193 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.532.244 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.532.262 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.532.277 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.532.317 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.532.334 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.532.356 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.179 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.180 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.181 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.187 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.185 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.564.188 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.319 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.341 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.361 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.358 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.385 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.377 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.404 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.411 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.396 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.421 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.427 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.426 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.466 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.497 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.469 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.474 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.486 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.507 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.515 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.529 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.533 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.546 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.552 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.561 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.570 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.584 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.593 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.610 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.616 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.639 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.564.648 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.658 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.664 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.681 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.693 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.564.704 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.916.183 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:50.916.221 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.916.259 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:50.916.277 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.916.335 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.916.356 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.916.397 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.971.423 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.471 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.971.498 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.971.548 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.586 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.614 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.716 [stars_engine.cc:643] 619447 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.780 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.802 [task_manager.cc:485] 619447 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.823 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.843 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.862 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.886 [stars_engine.cc:643] 619447 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.904 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.922 [task_manager.cc:485] 619447 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.935 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.952 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.965 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.971.998 [stars_engine.cc:643] 619447 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.016 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.032 [task_manager.cc:485] 619447 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.045 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.064 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.076 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.105 [stars_engine.cc:622] 619447 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=340, retCode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.121 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.137 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.972.154 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.972.188 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.212 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.226 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.242 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.258 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.271 [stream.cc:2886] 619447 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.287 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.300 [stream.cc:2886] 619447 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.314 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.327 [stream.cc:2886] 619447 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.344 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.356 [stream.cc:2886] 619447 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.370 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.382 [stream.cc:2886] 619447 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.396 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.409 [stream.cc:2886] 619447 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.438 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.452 [stream.cc:2886] 619447 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.469 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.483 [stream.cc:2886] 619447 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.497 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.510 [stream.cc:2886] 619447 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.523 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.537 [stream.cc:2886] 619447 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.554 [event.cc:310] 619447 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.583 [task_res.cc:380] 619447 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=7, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.610 [event.cc:249] 619447 UpdateLatestRecord: device_id=7, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.631 [event_task.cc:259] 619447 ConstructSqeForEventRecordTask: event_record: device_id=7, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.646 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.662 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.682 [task_submit.cc:573] 619447 AllocTaskAndSendStars: device_id=7, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.700 [stars_engine.cc:1122] 619447 SyncTask: Task Wait: device_id=7, stream_id=3, task_id=0, logicCqId=341. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.722 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.737 [stars_engine.cc:1071] 619447 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.755 [stars_engine.cc:1017] 619447 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.777 [stars_engine.cc:1027] 619447 ProcReport: taskType=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.790 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.804 [event.cc:249] 619447 UpdateLatestRecord: device_id=7, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.820 [event_task.cc:211] 619447 DoCompleteSuccessForEventRecordTask: event_record: device_id=7, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea570fbb92 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.836 [event.cc:122] 619447 TryFreeEventIdAndCheckCanBeDelete: device_id=7, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.865 [task_res.cc:323] 619447 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=7, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.894 [npu_driver.cc:5618] 619447 GetSqEnable: deviceId=7, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.911 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.972.924 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.972.936 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.972.987 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.007 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.022 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.033 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.049 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.064 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.077 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.092 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.106 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.119 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.136 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.148 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.180 [stars_engine.cc:622] 619447 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=341, retCode=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.195 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:50.973.208 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.221 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:50.973.238 [memory.cpp:421]619447 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.41s ============================================================================================================== 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340908] [debug] [619760] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340916] [debug] [619760] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340924] [debug] [619760] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340929] [debug] [619760] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360059] [debug] [619760] [lcal_comm.cpp:254] InitCommMem 7/8, localRank_ : 7, localRankSize_ : 8 success [2025-04-03 09:35:45.360193] [info] [619760] [lcal_comm.cpp:259] LcalCommInit 7/8 success and extraFlag:0 commArgs_.localRank : 7 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.390145] [error] [619794] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.395423] [error] [619794] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.398486] [error] [619794] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.277 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.326 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:50.982.409 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619447:281473672536096,MainProcess):2025-04-03-09:35:50.982.578 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.662 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.750 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.798 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:50.982.840 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.044.184 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.044.215 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.044.228 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.044.246 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.044.267 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.044.285 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.044.301 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.328 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.416 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619447,ffff51ae8fa0,python3.9):2025-04-03-09:35:51.101.495 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.582 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.606 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619447,ffff522f8fa0,python3.9):2025-04-03-09:35:51.101.651 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.724 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.748 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.101.804 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619447,ffff50ac8fa0,python3.9):2025-04-03-09:35:51.101.892 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.029 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.053 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619447,ffff512d8fa0,python3.9):2025-04-03-09:35:51.102.096 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.199 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.222 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.270 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.403 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.433 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.454 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.492 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.514 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.535 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.554 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.572 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.585 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.602 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.622 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.636 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.653 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.671 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.683 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.707 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.728 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.740 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.757 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.770 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.782 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.799 [stream.cpp:131]619447 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.813 [api_impl.cc:1394] 619447 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.827 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.843 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.102.856 [api_error.cc:895] 619447 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.102.872 [stream.cpp:146]619447 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.933 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.102.999 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.103.056 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.103.079 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.103.098 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.103.308 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.103.332 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.078 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.109 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.130 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.185 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.450 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.474 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.494 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.551 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.572 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.104.594 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_7 [INFO] GE_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.112.602 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.112.637 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.112.665 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.682 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.705 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.724 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.751 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.765 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.781 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.802 [device.cpp:143]619447 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.824 [api_impl.cc:2999] 619447 SetDevice: device_id=7, deviceMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.838 [device.cc:306] 619447 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.857 [runtime.cc:3393] 619447 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.872 [runtime.cc:3560] 619447 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.888 [api_impl.cc:2987] 619447 NewDevice: SetCurRef = 0xaaab0ca695c0,userDeviceId=7,deviceId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.901 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.923 [api_impl.cc:2992] 619447 NewDevice: New device success, user_device_id=7, die_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.112.941 [api_impl.cc:3025] 619447 SetDevice: SetDevice success, curCtx=0xaaab0ca695c0, user_device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.959 [acl.cpp:115]619447 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.975 [acl.cpp:121]619447 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.112.989 [device.cpp:154]619447 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 7 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:51.113.016 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:51.247.298 [awatchdog_monitor.c:160](tid:619479) thread 619794 not exist [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.184 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.189 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.272 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.342 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.333 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.354 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.351 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.366 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.396.369 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.377 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.391 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.448 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.473 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.477 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.485 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.499 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.483 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.494 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.508 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.514 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.542 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.525 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.530 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.533 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.550 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.557 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.576 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.585 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.592 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.614 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.621 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.624 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.636 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.648 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.653 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.664 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.396.709 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.717 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.722 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.729 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.734 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.396.745 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.428.182 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.428.222 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.428.242 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.428.257 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.428.271 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.428.288 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.428.306 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.532.761 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:51.532.794 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:51.532.813 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:51.532.852 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.532.873 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.532.943 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.556.169 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.556.195 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.556.209 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.556.244 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.556.258 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.556.280 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.556.294 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.940.169 [trs_cb_event.c:119][ascend][curpid:619447,619600][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =320; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:51.940.204 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.940.223 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:51.940.241 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.940.253 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:51.940.267 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:51.940.283 [npu_driver.cc:3828] 619600 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=320. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.068.173 [trs_cb_event.c:119][ascend][curpid:619447,619601][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=7, tid =321; ret=16) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.068.195 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.068.210 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.068.224 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.068.237 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.068.250 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.068.265 [npu_driver.cc:3828] 619601 CqReportIrqWait: Irq wait start, deviceId=7, tsId=0, groupId=321. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.181 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.178 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.183 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.187 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.206 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.294 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.228.238 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.312 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.324 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.335 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.349 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.350 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.357 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.365 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.384 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.376 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.383 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.393 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.399 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.411 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.412 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.423 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.433 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.442 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.453 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.467 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.472 [tensor_data_transfer.cpp:1054]619678 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is _npu_log [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.516 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.483 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.498 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.506 [tensor_data_transfer.cpp:1054]619684 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_image_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.524 [tensor_data_transfer.cpp:1056]619678 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.541 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.557 [tensor_data_transfer.cpp:1054]619686 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_scalar_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.575 [tensor_data_transfer.cpp:1056]619684 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.589 [api_impl.cc:6121] 619678 MemQueuePeek: Start to peek queue on devId 7, qid is 0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.228.608 [tensor_data_transfer.cpp:1056]619686 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.621 [api_impl.cc:6121] 619684 MemQueuePeek: Start to peek queue on devId 7, qid is 3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.633 [npu_driver.cc:4330] 619678 MemQueuePeek: peek mem queue, deviceId=7, qid=0, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.645 [api_impl.cc:6121] 619686 MemQueuePeek: Start to peek queue on devId 7, qid is 4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.655 [npu_driver.cc:4330] 619684 MemQueuePeek: peek mem queue, deviceId=7, qid=3, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.228.678 [npu_driver.cc:4330] 619686 MemQueuePeek: peek mem queue, deviceId=7, qid=4, timeout=800. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.342 [op_base.cc:2136] [619447]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619447,fffeb37eefa0,python3.9):2025-04-03-09:35:52.340.360 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.340.376 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.397 [adapter_rts.cc:198] [619447][hrtGetDeviceRefresh]deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.416 [op_base.cc:2151] [619447][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.442 [op_base.cc:128] [619447][HcclGetDeviceId] deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.477 [op_base.cc:2168] [619447]com is not global com [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.500 [hcom_plugin.cc:381] [619447][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.550 [hccl_comm.cc:1375] [619447]UnRegistTaskAbortHandler begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.570 [task_abort_handler.cc:140] [619447]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.340.588 [api_impl.cc:4192] 619447 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.340.602 [runtime.cc:4698] 619447 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.620 [hccl_communicator.cc:2112] [619447][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.340.639 [hccl_communicator.cc:2114] [619447][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.655 [hccl_communicator.cc:2082] [619447]start to stop the backGround Thread [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.340.670 [heartbeat.cc:1203] [619447]Heartbeat has been destroyed [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.698 [heartbeat.cc:451] [619447][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.340.712 [heartbeat.cc:452] [619447]Heartbeat has been destroyed [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.729 [hccl_impl.cc:69] [619447]start hccl resources destruction:deviceLogicId[7] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.340.746 [heartbeat.cc:1203] [619447]Heartbeat has been destroyed [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.340.761 [heartbeat.cc:451] [619447][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.340.774 [heartbeat.cc:452] [619447]Heartbeat has been destroyed [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.340.887 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.341.049 [task_exception_handler.cc:1232] [619447]deInit taskFailCallback [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.342.101 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.342.125 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.342.168 [api_error.cc:1278] 619447 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.342.187 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.221 [hccl_socket_manager.cc:81] [619447][DeInit][Server]ip[192.168.2.192] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.241 [hccl_socket_manager.cc:83] [619447][DeInit][Server]ip[192.168.2.192] port[16666] [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.342.253 [hccl_socket.cc:232] [619447][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.267 [network_manager.cc:69] [619447]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.310 [network_manager.cc:458] [619447]NetworkManager devicePhyId_[7] vnic stopped. refcount[0] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:52.342.355 [ra_host.c:911]tid:619447,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[7], local_ip[192.168.2.192] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:52.342.695 [ra_host.c:404]tid:619447,ra_socket_deinit(404) : Input parameters: phy_id[7] family[2] local_ip[192.168.2.192] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.855 [hccl_socket.cc:68] [619447][HcclSocket][DeInit] device[7] stops listen on ip[192.168.2.192], port[16666] success, socketType[0]. [WARNING] HCCL(619447,python3.9):2025-04-03-09:35:52.342.878 [hccl_socket.cc:232] [619447][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.896 [hccl_communicator.cc:1741] [619447][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.342.913 [network_manager.cc:69] [619447]NetworkManager::GetInstance deviceLogicID[7]. [INFO] HCCP(619447,python3.9):2025-04-03-09:35:52.342.928 [ra_host.c:248]tid:619447,ra_deinit(248) : Input parameters: phy_id[7], nic_position:[1] [INFO] HCCP(619447,python3.9):2025-04-03-09:35:52.342.941 [ra_hdc.c:651]tid:619447,ra_hdc_deinit(651) : hdc deinit start! phy_id[7] [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.343.059 [hdc_client.c:417][ascend][curpid:619447,619447][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4274) [INFO] ROCE(619447,python3.9):2025-04-03-09:35:52.343.158 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619447,python3.9):2025-04-03-09:35:52.343.175 [ra_hdc.c:685]tid:619447,ra_hdc_deinit(685) : hdc deinit OK! phy_id[7] [INFO] ROCE(619447,python3.9):2025-04-03-09:35:52.343.207 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.289 [tsd_client.cpp:189][TsdCapabilityGet][tid:619447] TsdCapabilityGet Begin. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.307 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.322 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.339 [process_mode_manager.cpp:1042][CapabilityGet][tid:619447] [ProcessModeManager] enter into CapabilityGet process deviceId[7] type[6] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.343.354 [adapter_tdt.cc:66] [619447]Get TsdCapability success. deviceLogicId[7] [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.367 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.379 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.392 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619447] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.410 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619447] add close subproc:31012, proctype:0 [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.343.448 [hdc_client.cpp:154][TsdRecvData][tid:619447] recv timeout is:150000 ms [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.415.306 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619447] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.415.329 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619447] [TsdClient] DeviceMsgProc recvMsg realDeviceId[7] msgType[61] localDevId[7] rspCode[0]heterogeneousSubPid[31012], tsdSupportLevel_[63] [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.415.349 [hdc_client.cpp:172][TsdRecvData][tid:619447] [deviceId=7] the Tsd recv data already [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.415.365 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619447] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.415.380 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619447] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.395 [adapter_tdt.cc:52] [619447]Open TsdProcess success. deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.406 [network_manager.cc:405] [619447][DeInit]finish CloseHccpProcess, phy_id[7], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.421 [network_manager.cc:409] [619447]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.435 [hccl_communicator_attrs.cc:1307] [619447][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.449 [hccl_communicator_attrs.cc:1275] [619447]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.466 [notify_pool_impl.cc:43] [619447]NotifyPoolImpl Destroy. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.486 [notify_pool_impl.cc:56] [619447]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.507 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.521 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.536 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.570 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.587 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.603 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.615 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.631 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.655 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.672 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.687 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=0. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.415.712 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=0.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.729 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.742 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.754 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.769 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.784 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.800 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.816 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.834 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=1. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.415.852 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=1.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.868 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.881 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.894 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.909 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.925 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.941 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.415.954 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.415.967 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=2. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.415.985 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=2.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.003 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.015 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.043 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.056 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.067 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.081 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.096 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.110 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=3. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.416.127 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=3.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.146 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.167 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.182 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.197 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.213 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.229 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.244 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.262 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=4. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.416.277 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=4.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.292 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.302 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.315 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.327 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.340 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.353 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.367 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.383 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=5. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.416.396 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=5.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.413 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.428 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.441 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.453 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.475 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.490 [api_impl.cc:3520] 619447 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.504 [p2p_mgmt.cc:178] [619447]disable p2p: local logic id:7, remote physic id:6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.518 [api_impl.cc:3184] 619447 DisableP2P: Disable P2P devIdDes=7, phyIdSrc=6. [INFO] DRV(619447,python3.9):2025-04-03-09:35:52.416.534 [devdrv_pcie.c:890][ascend][curpid:619447,619447][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=7; peer_phy_devid=6.) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.552 [api_impl.cc:3100] 619447 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.564 [api_impl.cc:3111] 619447 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.577 [api_impl.cc:3130] 619447 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.591 [p2p_mgmt.cc:171] [619447]local logic id:7, local physic id:7. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.608 [hccl_communicator.cc:2043] [619447]Profiling is deinitiated. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.622 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.648 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.416.665 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.682 [opexecounter.cc:103] [619447]free counter mem resource [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.416.698 [atrace_api.c:118](tid:619447) AtraceDestroy start. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.416.716 [tracer_mgr_operate.c:248](tid:619447) destroy object HCCL_619760_7, exitSave(true). [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.416.731 [atrace_api.c:120](tid:619447) AtraceDestroy end. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.416.741 [adapter_trace.cc:33] [619447]Call TraceDestroy, Params:handle[0xaaaafbe0bb08] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.418.731 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.418.772 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.418.801 [queue_notify_manager.cc:100] [619447]QueueNotifyManager Destroy. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.418.819 [queue_notify_manager.cc:106] [619447]QueueNotifyManager Destroy success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.418.947 [queue_notify_manager.cc:100] [619447]QueueNotifyManager Destroy. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.418.966 [queue_notify_manager.cc:106] [619447]QueueNotifyManager Destroy success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.035 [ccl_buffer_manager.cc:125] [619447][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.055 [ccl_buffer_manager.cc:147] [619447][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.072 [ccl_buffer_manager.cc:160] [619447][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.085 [ccl_buffer_manager.cc:163] [619447][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.101 [ccl_buffer_manager.cc:166] [619447][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.136 [op_base.cc:128] [619447][HcclGetDeviceId] deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.165 [op_base.cc:2196] [619447]op_base comm destroy complete, take time [78790]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[7]. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.419.183 [device.cpp:183]619447 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.202 [api_impl.cc:3419] 619447 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.214 [runtime.cc:3444] 619447 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.230 [api_impl.cc:3449] 619447 DeviceReset: Succ, userDeviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.419.241 [device.cpp:189]619447 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 7. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.419.282 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.419.312 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.419.327 [hcom_executor.cc:40] [619447]Hcom Excutor Finalize start. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.383 [api_impl.cc:3419] 619704 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.412 [runtime.cc:3444] 619704 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.432 [api_impl.cc:3449] 619704 DeviceReset: Succ, userDeviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.491 [api_impl.cc:3419] 619702 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.519 [runtime.cc:3444] 619702 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.537 [api_impl.cc:3449] 619702 DeviceReset: Succ, userDeviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.689 [api_impl.cc:3419] 619703 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.715 [runtime.cc:3444] 619703 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.729 [api_impl.cc:3449] 619703 DeviceReset: Succ, userDeviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.798 [api_impl.cc:3419] 619705 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.835 [runtime.cc:3444] 619705 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.852 [api_impl.cc:3449] 619705 DeviceReset: Succ, userDeviceId=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.419.986 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.420.009 [adapter_rts.cc:321] [619447][hrtCtxSetCurrent] success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.420.023 [stream.cc:182] [619447]Switch Ctx ret[0], curCtx[0xaaab0ca695c0], setCtx[0xaaab0ca695c0], stream id[5] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.047 [api_impl.cc:1228] 619447 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.420.071 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.420.118 [single_op_manager.cc:50]619447 ReleaseResource:ReleaseResource in. resource id = 0xaaab17e0c180 [INFO] GE(619447,python3.9):2025-04-03-09:35:52.420.145 [caching_allocator.cc:402]619447 TryFreeBlocks:Try free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.420.223 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:52.420.267 [npu_memory_allocator.cc:72]619447 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.420.286 [stream_executor.cpp:197]619447 RemoveExecutor: To remove executor by stream = 187651816735104 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.420.323 [acl_resource_manager.cpp:823]619447 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.341 [stream.cc:852] 619447 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.358 [stream.cc:938] 619447 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.373 [stream.cc:1507] 619447 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.407 [stream_sqcq_manage.cc:451] 619447 FreeLogicCq: Return(0), threadIdentifier(281473672536096), devId(7), tsId(0), cqId(343), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.437 [npu_driver.cc:6163] 619447 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=5, logicCq=343. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.420.479 [npu_driver.cc:1180] 619447 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=343, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.253 [stream.cc:237] 619447 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.286 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.303 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.948 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.970 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.424.986 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.008 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=5, device_id=7, tsId=0, streamFlag=1536 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.425.668 [stream.cc:185] [619447][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab17e0c180], stream id[5] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.689 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.425.705 [adapter_rts.cc:321] [619447][hrtCtxSetCurrent] success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.425.719 [stream.cc:193] [619447]Restore Ctx ret[0], setCtx[0xaaab0ca695c0], stream id[5] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.775 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.425.794 [adapter_rts.cc:321] [619447][hrtCtxSetCurrent] success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.425.808 [stream.cc:182] [619447]Switch Ctx ret[0], curCtx[0xaaab0ca695c0], setCtx[0xaaab0ca695c0], stream id[4] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.822 [api_impl.cc:1228] 619447 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.425.838 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.425.853 [single_op_manager.cc:50]619447 ReleaseResource:ReleaseResource in. resource id = 0xaaab17d90120 [INFO] GE(619447,python3.9):2025-04-03-09:35:52.425.890 [caching_allocator.cc:402]619447 TryFreeBlocks:Try free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.425.903 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:52.425.917 [npu_memory_allocator.cc:72]619447 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.425.929 [stream_executor.cpp:197]619447 RemoveExecutor: To remove executor by stream = 187651816227104 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.425.942 [acl_resource_manager.cpp:823]619447 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.953 [stream.cc:852] 619447 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.969 [stream.cc:938] 619447 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.425.979 [stream.cc:1507] 619447 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.426.001 [stream_sqcq_manage.cc:451] 619447 FreeLogicCq: Return(0), threadIdentifier(281473672536096), devId(7), tsId(0), cqId(342), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.426.022 [npu_driver.cc:6163] 619447 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=4, logicCq=342. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.426.056 [npu_driver.cc:1180] 619447 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=342, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.431 [stream.cc:237] 619447 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.454 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.470 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.958 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.979 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.429.995 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.430.012 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=4, device_id=7, tsId=0, streamFlag=1536 [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.614 [stream.cc:185] [619447][HCCL_TRACE]StreamDestroy, streamPtr[0xaaab17d90120], stream id[4] [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.430.636 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.651 [adapter_rts.cc:321] [619447][hrtCtxSetCurrent] success [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.663 [stream.cc:193] [619447]Restore Ctx ret[0], setCtx[0xaaab0ca695c0], stream id[4] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.691 [hcom_executor.cc:45] [619447][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.430.731 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.430.758 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.780 [hcom_ops_kernel_info_store.cc:3423] [619447]finalize hccl kernel info store. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.817 [hcom_common.cc:85] [619447][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[7] [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.839 [plugin_manager.cc:56] [619447]hccl ops plugin finalize start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.853 [hcom_plugin.cc:90] [619447]Finalize start. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.937 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.957 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.972 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.430.985 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.016 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.032 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.045 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.057 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.083 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.100 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.113 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.124 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.155 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.169 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.179 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.192 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.222 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.235 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.246 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.258 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.287 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.300 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.311 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.324 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.334 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.346 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.357 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.368 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.382 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.392 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.413 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.426 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.456 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.473 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.485 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.496 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.522 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.535 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.549 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.563 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.592 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.607 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.620 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.630 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.663 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.679 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.692 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.703 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.730 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.744 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.755 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.768 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.797 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.811 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.822 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.833 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.862 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.875 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.887 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.898 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.924 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.938 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.949 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.960 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.983 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.431.998 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.008 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.021 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.032 [hcom_plugin.cc:119] [619447]Finalize: HcomDestroy success. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.044 [plugin_manager.cc:62] [619447]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.432.102 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.432.127 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.141 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.164 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.184 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.197 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.210 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.223 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.237 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.249 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.263 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.274 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.284 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.297 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.309 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.320 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.331 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.343 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.353 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.365 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.379 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.390 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.399 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.412 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.422 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.432 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.457 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.469 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.481 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.491 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.503 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.517 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.528 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.538 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.550 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.563 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.574 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.588 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.599 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.613 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.624 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.634 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.647 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.659 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.669 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.681 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.690 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.701 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.711 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.724 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.735 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.748 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.760 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.772 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.786 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.797 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.807 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.821 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.835 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.863 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.874 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.885 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.894 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.907 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.920 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.931 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.943 [hcom_common.cc:958] [619447][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.956 [hcom_common.cc:961] [619447]Entry-HcomDestroy:void skip [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.966 [hcom.cc:337] [619447][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:52.432.981 [hcom.cc:340] [619447]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.004 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.023 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.204 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.228 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.245 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.260 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.275 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.290 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.303 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61027123200 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.316 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.334 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61027122688, total count : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.350 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.364 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.391 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.405 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.419 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61027123200 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.439 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.453 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61027122688, total count : 1 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.466 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116400.00MB,"idle mem":116399.99MB,"used by event mem":0.00MB,"eager free mem":116399.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.493 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116400.00MB,"idle mem":116399.99MB,"used by event mem":0.00MB,"eager free mem":116399.99MB}. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.511 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.528 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.548 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.565 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.583 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.596 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.611 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.630 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.646 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.672 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.690 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.709 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61027123200,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.725 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.742 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.762 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffeac100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.778 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffeac101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.433.799 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58200M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.819 [callback.cpp:37]619447 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.839 [callback.cpp:44]619447 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.872 [context.cpp:72]619447 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.433.889 [api_impl.cc:3717] 619447 ContextSetCurrent: set context=aaab0ca695c0, flag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.901 [context.cpp:80]619447 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.915 [stream.cpp:75]619447 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619447,python3.9):2025-04-03-09:35:52.433.934 [stream_manage_func_registry.cc:22]619447 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.949 [stream.cpp:80]619447 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.433.964 [api_impl.cc:1228] 619447 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.433.977 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.433.990 [single_op_manager.cc:50]619447 ReleaseResource:ReleaseResource in. resource id = 0xaaab0d9d25f0 [INFO] GE(619447,python3.9):2025-04-03-09:35:52.434.006 [caching_allocator.cc:402]619447 TryFreeBlocks:Try free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.434.018 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:52.434.041 [npu_memory_allocator.cc:72]619447 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.054 [stream_executor.cpp:197]619447 RemoveExecutor: To remove executor by stream = 187651644532208 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.070 [acl_resource_manager.cpp:823]619447 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.083 [stream.cc:852] 619447 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.097 [stream.cc:938] 619447 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.112 [event.cc:1036] 619447 WaitForBusy: device_id=7, event_id=0 finished map already empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.129 [npu_driver.cc:626] 619447 EventIdFree: event id free begin, device_id=7, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.152 [raw_device.cc:1158] 619447 FreeEventIdFromDrv: device_id=7, event_id=0 tear down. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.164 [event.cc:61] 619447 ~Event: device_id=7, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.179 [npu_driver.cc:3803] 619447 SqCqFree: deviceId=7, tsId=0, sqId=118, cqId=118 [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.434.219 [trs_sqcq.c:1491][ascend][curpid:619447,619600][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.220 [subscribe.cc:244] 619447 Delete: delete callback subscribe: groupId=320, threadId=281471797227424, devId=7, tsId=0, streamId=2, sqId=118, cqId=118 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.237 [npu_driver.cc:3833] 619600 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=320, drvRetCode=16. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.245 [stream.cc:1507] 619447 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.251 [api_impl.cc:5058] 619600 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.264 [callback.cpp:132]619600 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.269 [stream_sqcq_manage.cc:451] 619447 FreeLogicCq: Return(0), threadIdentifier(281473672536096), devId(7), tsId(0), cqId(340), isFastCq(0). [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.278 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.286 [npu_driver.cc:6163] 619447 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=2, logicCq=340. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.293 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.308 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.326 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.331 [npu_driver.cc:1180] 619447 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=340, drvFlag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.342 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.358 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.370 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.395 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.407 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.421 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.431 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.441 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.452 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.463 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.475 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.488 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.500 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.510 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.523 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.536 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.546 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.556 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.567 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.581 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.595 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.607 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.619 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.632 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.644 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.654 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.664 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.688 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.699 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.709 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.719 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.734 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.746 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.757 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.767 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.779 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.790 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.803 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.816 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.828 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.841 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.854 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.868 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.879 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.891 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.903 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.916 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.929 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.941 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.953 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.434.964 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.434.979 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.000 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.012 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.025 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.039 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.052 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.062 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.076 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.089 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.100 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.111 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.123 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.135 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.146 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.158 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.169 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.182 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.193 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.204 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.216 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.227 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.238 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.250 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.262 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.279 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.291 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.311 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.322 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.335 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.348 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.359 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.369 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.382 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.394 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.406 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.416 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.429 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.440 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.451 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.464 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.475 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.490 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.505 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.518 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.529 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.540 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.555 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.568 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.581 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.592 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.602 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.629 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.643 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.655 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.668 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.680 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.692 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.702 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.714 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.726 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.736 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.747 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.761 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.772 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.784 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.795 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.806 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.816 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.830 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.841 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.852 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.863 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.874 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.884 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.900 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.910 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.933 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.944 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.954 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.435.967 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.980 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.435.991 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.003 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.015 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.031 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.044 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.055 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.065 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.075 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.088 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.103 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.114 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.123 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.133 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.147 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.173 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.191 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.202 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.213 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.224 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.245 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.259 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.270 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.281 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.292 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.302 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.316 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.328 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.341 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.351 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.362 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.376 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.390 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.405 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.419 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.432 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.443 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.458 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.474 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.485 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.497 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.507 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.520 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.534 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.545 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.561 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.575 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.586 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.599 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.610 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.623 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.633 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.643 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.655 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.668 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.678 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.689 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.700 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.710 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.722 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.736 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.746 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.759 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.769 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.780 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.790 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.802 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.812 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.824 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.835 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.857 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.871 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.883 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.894 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.909 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.921 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.933 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.948 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.960 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.436.971 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.982 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.436.992 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.005 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.015 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.027 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.036 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.047 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.057 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.071 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.083 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.094 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.106 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.119 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.130 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.143 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.163 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.175 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.187 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.199 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.210 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.225 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.240 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.254 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.264 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.277 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.287 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.301 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.311 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.323 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.333 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.347 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.360 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.360 [stream.cc:237] 619447 FreeStreamId: Free stream_id=2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.372 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.384 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.387 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.396 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=7, tsId=0, drvFlag=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.400 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.413 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.425 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.446 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.459 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.471 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.484 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.499 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.514 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.529 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.541 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.552 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.564 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.575 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.587 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.599 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.610 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.621 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.633 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.645 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.659 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.670 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.683 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.694 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.706 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.717 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.731 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.744 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.764 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.776 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.787 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.803 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.815 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.828 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.840 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.852 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.865 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.877 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.887 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.885 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=7, tsId=0, drvFlag=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.898 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.908 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.912 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.923 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=2, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.931 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.939 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=2, device_id=7, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.943 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.961 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.972 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.437.983 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.437.998 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.010 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.030 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.041 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.052 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.063 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.075 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.090 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.105 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.118 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.131 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.141 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.153 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.165 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.176 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.187 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.198 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.209 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.221 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.234 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.246 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.258 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.269 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.279 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.290 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.301 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.315 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.337 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.349 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.358 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.370 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.382 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.394 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.405 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.420 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.429 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.440 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.451 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.467 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.480 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.492 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.503 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.519 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.534 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.548 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.561 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.572 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.582 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.594 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.608 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.621 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.633 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.658 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.671 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.686 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.696 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.707 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.718 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.730 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.742 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.756 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.767 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.779 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.791 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.801 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.812 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.827 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.839 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.850 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.864 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.876 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.888 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.899 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.912 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.926 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.935 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.956 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.438.969 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.982 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.438.995 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.006 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.016 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.028 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.038 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.051 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.064 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.075 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.085 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.096 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.108 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.120 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.134 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.145 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.155 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.166 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.178 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.189 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.203 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.215 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.227 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.238 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.257 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.273 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.286 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.296 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.307 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.320 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.330 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.341 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.355 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.366 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.378 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.392 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.405 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.417 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.430 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.440 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.452 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.464 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.475 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.488 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.501 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.512 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.523 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.533 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.545 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.569 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.583 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.594 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.604 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.615 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.625 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.636 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.648 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.659 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.672 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.683 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.693 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.708 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.721 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.732 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.744 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.755 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.765 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.777 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.791 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.805 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.818 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.830 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.839 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.852 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.871 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.882 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.892 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.903 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.916 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.928 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.943 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.953 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.439.963 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.978 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.439.990 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.007 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.021 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.032 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.042 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.053 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.065 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.078 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.089 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.101 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.112 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.124 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.135 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.148 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.178 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.195 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.205 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.216 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.226 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.239 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.253 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.264 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.275 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.285 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.296 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.308 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.319 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.329 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.341 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.352 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.364 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.376 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.386 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.399 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.411 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.420 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.433 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.447 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.458 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.482 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.493 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.504 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.516 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.529 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.543 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.557 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.567 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.577 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.589 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.602 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.613 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.627 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.637 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.648 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.658 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.670 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.682 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.694 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.706 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.717 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.729 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.742 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.755 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.767 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.788 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.800 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.812 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.826 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.841 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.855 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.869 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.880 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.892 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.905 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.916 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.929 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.942 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.955 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.440.967 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.979 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.989 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.440.998 [stream.cpp:87]619447 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.003 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.028 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.039 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.441.044 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab0d9d25f0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.053 [callback.cpp:116]619600 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.064 [stream.cpp:75]619447 aclrtDestroyStream: start to execute aclrtDestroyStream [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.067 [subscribe.cc:308] 619600 GetGroupIdByThreadId: can not find subscribe info, threadId=281471797227424 [INFO] GE(619447,python3.9):2025-04-03-09:35:52.441.083 [stream_manage_func_registry.cc:22]619447 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.111 [api_impl.cc:5043] 619600 ProcessReport: get groupId fail, threadIdentifier=281471797227424, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.119 [stream.cpp:80]619447 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.127 [callback.cpp:130]619600 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.137 [api_impl.cc:1228] 619447 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.154 [acl_resource_manager.cpp:814]619447 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.441.167 [single_op_manager.cc:50]619447 ReleaseResource:ReleaseResource in. resource id = 0xaaab0db033f0 [INFO] GE(619447,python3.9):2025-04-03-09:35:52.441.180 [caching_allocator.cc:402]619447 TryFreeBlocks:Try free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.441.192 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] DEVICE(619447,ffff427cefa0,python3.9):2025-04-03-09:35:52.441.187 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] GE(619447,python3.9):2025-04-03-09:35:52.441.206 [npu_memory_allocator.cc:72]619447 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.218 [stream_executor.cpp:197]619447 RemoveExecutor: To remove executor by stream = 187651645780976 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.232 [acl_resource_manager.cpp:823]619447 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.244 [stream.cc:852] 619447 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.258 [stream.cc:938] 619447 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.269 [event.cc:1036] 619447 WaitForBusy: device_id=7, event_id=1 finished map already empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.283 [npu_driver.cc:626] 619447 EventIdFree: event id free begin, device_id=7, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.304 [raw_device.cc:1158] 619447 FreeEventIdFromDrv: device_id=7, event_id=1 tear down. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.317 [event.cc:61] 619447 ~Event: device_id=7, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.332 [npu_driver.cc:3803] 619447 SqCqFree: deviceId=7, tsId=0, sqId=119, cqId=119 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.362 [subscribe.cc:244] 619447 Delete: delete callback subscribe: groupId=321, threadId=281471788773280, devId=7, tsId=0, streamId=3, sqId=119, cqId=119 [WARNING] DRV(619447,python3.9):2025-04-03-09:35:52.441.366 [trs_sqcq.c:1491][ascend][curpid:619447,619601][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=7; type=1; ret=16) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.379 [stream.cc:1507] 619447 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.387 [npu_driver.cc:3833] 619601 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=7, tsId=0, groupId=321, drvRetCode=16. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.403 [api_impl.cc:5058] 619601 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.404 [stream_sqcq_manage.cc:451] 619447 FreeLogicCq: Return(0), threadIdentifier(281473672536096), devId(7), tsId(0), cqId(341), isFastCq(0). [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.416 [callback.cpp:132]619601 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.445 [npu_driver.cc:6163] 619447 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=3, logicCq=341. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.449 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.464 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.475 [npu_driver.cc:1180] 619447 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=341, drvFlag=0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.476 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.493 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.505 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.517 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.529 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.539 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.550 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.561 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.573 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.588 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.599 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.611 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.621 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.634 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.646 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.656 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.666 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.676 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.688 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.720 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.731 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.743 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.755 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.770 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.784 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.795 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.806 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.817 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.829 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.843 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.854 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.866 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.876 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.887 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.898 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.912 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.925 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.938 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.949 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.961 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.441.972 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.986 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.441.995 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.006 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.027 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.039 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.052 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.067 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.080 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.093 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.106 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.120 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.130 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.141 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.150 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.162 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.172 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.182 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.195 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.205 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.214 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.225 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.237 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.249 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.265 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.277 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.289 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.299 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.309 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.330 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.342 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.354 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.365 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.376 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.388 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.401 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.413 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.423 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.433 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.443 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.455 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.470 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.481 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.492 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.501 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.513 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.526 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.539 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.549 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.561 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.576 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.590 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.600 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.611 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.629 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.642 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.653 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.665 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.677 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.688 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.700 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.711 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.722 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.733 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.745 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.760 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.774 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.786 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.797 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.813 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.826 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.837 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.847 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.861 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.873 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.887 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.898 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.908 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.928 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.941 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.952 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.963 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.442.973 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.986 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.442.999 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.010 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.021 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.032 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.042 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.052 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.065 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.077 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.087 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.098 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.110 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.122 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.135 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.148 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.158 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.168 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.179 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.190 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.205 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.224 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.234 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.246 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.260 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.272 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.282 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.292 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.303 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.315 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.329 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.340 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.353 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.366 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.378 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.389 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.400 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.415 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.426 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.436 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.445 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.459 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.472 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.486 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.495 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.506 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.528 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.543 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.553 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.562 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.572 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.584 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.596 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.610 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.621 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.632 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.643 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.656 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.668 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.681 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.693 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.707 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.719 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.731 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.742 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.753 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.767 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.780 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.792 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.805 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.814 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.833 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.848 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.860 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.871 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.882 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.893 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.907 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.919 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.931 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.941 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.951 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.963 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.974 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.443.986 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.443.999 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.010 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.022 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.034 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.047 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.057 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.067 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.079 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.091 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.105 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.116 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.133 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.144 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.172 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.188 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.199 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.213 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.226 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.241 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.252 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.264 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.276 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.284 [stream.cc:237] 619447 FreeStreamId: Free stream_id=3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.292 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.299 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.305 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.311 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=7, tsId=0, drvFlag=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.318 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.334 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.348 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.361 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.376 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.388 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.399 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.409 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.421 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.444 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.455 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.468 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.481 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.491 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.502 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.513 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.526 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.540 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.551 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.561 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.577 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.590 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.600 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.610 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.625 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.636 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.646 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.657 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.672 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.685 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.696 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.707 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.722 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.733 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.750 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.764 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.776 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.786 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.791 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=7, tsId=0, drvFlag=0 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.796 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.809 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.814 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.826 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.831 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.841 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=3, device_id=7, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.843 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.858 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.871 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.886 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.897 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.914 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.927 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.938 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.949 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.961 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.444.975 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.444.989 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.000 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.023 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.035 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.046 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.060 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.072 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.083 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.094 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.106 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.118 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.129 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.140 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.153 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.165 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.176 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.191 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.202 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.213 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.225 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.240 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.252 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.263 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.277 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.290 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.300 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.315 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.335 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.350 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.367 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.380 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.391 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.405 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.416 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.428 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.442 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.454 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.464 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.475 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.488 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.490 [event.cc:61] 619447 ~Event: device_id=7, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.499 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.510 [stream.cpp:87]619447 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.511 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.532 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.542 [callback.cpp:116]619601 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.547 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaab0db033f0. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.553 [subscribe.cc:308] 619601 GetGroupIdByThreadId: can not find subscribe info, threadId=281471788773280 [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.568 [device.cpp:183]619447 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 7. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.570 [api_impl.cc:5043] 619601 ProcessReport: get groupId fail, threadIdentifier=281471788773280, reCode=0x7160004 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.587 [api_impl.cc:3419] 619447 DeviceReset: reset device, device_id=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.589 [callback.cpp:130]619601 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.605 [runtime.cc:3444] 619447 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.445.627 [api_impl.cc:3449] 619447 DeviceReset: Succ, userDeviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.641 [device.cpp:189]619447 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 7. [INFO] DEVICE(619447,ffff41fbefa0,python3.9):2025-04-03-09:35:52.445.660 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.667 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.724 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.758 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.776 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.789 [callback.cpp:37]619447 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:52.445.808 [callback.cpp:44]619447 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.829 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:52.445.852 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.587.359 [awatchdog_monitor.c:160](tid:619479) thread 619702 not exist [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.587.396 [awatchdog_monitor.c:160](tid:619479) thread 619704 not exist [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.587.417 [awatchdog_monitor.c:160](tid:619479) thread 619703 not exist [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:52.587.437 [awatchdog_monitor.c:160](tid:619479) thread 619705 not exist [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.744.713 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.744.740 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.744.757 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:52.744.772 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.744.788 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:52.744.860 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.056.176 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.056.177 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.056.181 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.056.181 [queue_client_comm.c:114][ascend][curpid:619447,619678][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.303 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.324 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.328 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.341 [tensor_data_transfer.cpp:1054]619680 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.348 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.356 [tensor_data_transfer.cpp:1056]619680 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.363 [tensor_data_transfer.cpp:1054]619682 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.364 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.369 [api_impl.cc:6121] 619680 MemQueuePeek: Start to peek queue on devId 7, qid is 1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.379 [tensor_data_transfer.cpp:1056]619682 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.393 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.396 [npu_driver.cc:4338] 619678 MemQueuePeek: queue empty [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.404 [npu_driver.cc:4330] 619680 MemQueuePeek: peek mem queue, deviceId=7, qid=1, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.412 [api_impl.cc:6121] 619682 MemQueuePeek: Start to peek queue on devId 7, qid is 2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.424 [tensor_data_transfer.cpp:1054]619690 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 7, name is ms_histogram_summary [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.432 [tensor_data_transfer.cpp:612]619678 acltdtReceiveTensorV2: queue is empty, device is 7, name is _npu_log [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.466 [tensor_data_transfer.cpp:1056]619690 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.453 [npu_driver.cc:4330] 619682 MemQueuePeek: peek mem queue, deviceId=7, qid=2, timeout=800. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.481 [api_impl.cc:6121] 619690 MemQueuePeek: Start to peek queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.503 [npu_driver.cc:4330] 619690 MemQueuePeek: peek mem queue, deviceId=7, qid=5, timeout=800. [INFO] DEVICE(619447,ffff1047dfa0,python3.9):2025-04-03-09:35:53.056.610 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.056.749 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is _npu_log [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.778 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.056.796 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=0. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.057.223 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is _npu_log [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.057.267 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.057.293 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.060.172 [queue_client_comm.c:114][ascend][curpid:619447,619684][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.060.185 [queue_client_comm.c:114][ascend][curpid:619447,619686][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.060.276 [npu_driver.cc:4338] 619684 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.060.296 [tensor_data_transfer.cpp:612]619684 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_image_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.060.304 [npu_driver.cc:4338] 619686 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.060.324 [tensor_data_transfer.cpp:612]619686 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_scalar_summary [INFO] DEVICE(619447,fffef2fdefa0,python3.9):2025-04-03-09:35:53.060.410 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619447,fffef1fbefa0,python3.9):2025-04-03-09:35:53.060.478 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.888.174 [queue_client_comm.c:114][ascend][curpid:619447,619680][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.888.175 [queue_client_comm.c:114][ascend][curpid:619447,619682][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619447,python3.9):2025-04-03-09:35:53.888.196 [queue_client_comm.c:114][ascend][curpid:619447,619690][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.888.267 [npu_driver.cc:4338] 619680 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.888.284 [tensor_data_transfer.cpp:612]619680 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_dump [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.888.292 [npu_driver.cc:4338] 619682 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.888.307 [tensor_data_transfer.cpp:612]619682 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_tensor_summary [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:53.888.417 [npu_driver.cc:4338] 619690 MemQueuePeek: queue empty [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.888.453 [tensor_data_transfer.cpp:612]619690 acltdtReceiveTensorV2: queue is empty, device is 7, name is ms_histogram_summary [INFO] DEVICE(619447,fffef3ffefa0,python3.9):2025-04-03-09:35:53.888.461 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619447,ffff093e8fa0,python3.9):2025-04-03-09:35:53.888.465 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619447,fffef0f9efa0,python3.9):2025-04-03-09:35:53.888.645 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.888.886 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is ms_histogram_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.888.916 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.888.929 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=5. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.889.339 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is ms_histogram_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.889.377 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.889.403 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.889.451 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is ms_scalar_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.889.472 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 4. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.889.487 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=4. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.889.843 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is ms_scalar_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.889.873 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.889.895 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.889.964 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is ms_image_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.889.985 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 3. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.890.000 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=3. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.890.433 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is ms_image_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.890.464 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.890.498 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.890.548 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is ms_tensor_summary [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.890.569 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.890.581 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=2. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.890.955 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is ms_tensor_summary [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.890.985 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.008 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.891.051 [tensor_data_transfer.cpp:973]619447 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 7, name is ms_tensor_dump [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.891.072 [api_impl.cc:6065] 619447 MemQueueDestroy: Start to destroy queue on devId 7, qid is 1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.891.084 [npu_driver.cc:3989] 619447 MemQueueDestroy: deviceId=7, qid=1. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.891.438 [tensor_data_transfer.cpp:977]619447 acltdtDestroyChannel: acltdtDestroyChannel success, device is 7, name is ms_tensor_dump [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.891.462 [device.cpp:183]619447 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.891.483 [api_impl.cc:3419] 619447 DeviceReset: reset device, device_id=7. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.891.498 [runtime.cc:3444] 619447 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.891.516 [api_impl.cc:3449] 619447 DeviceReset: Succ, userDeviceId=7. [INFO] ASCENDCL(619447,python3.9):2025-04-03-09:35:53.891.526 [device.cpp:189]619447 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 7. [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.552 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.582 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.628 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.648 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.671 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_7 [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.691 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.714 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.733 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.891.774 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.891.933 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.891.967 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.891.988 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.008 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.035 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.065 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.091 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] PS(619447,ffff437eefa0,python3.9):2025-04-03-09:35:53.892.111 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] DISTRIBUTED(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.112 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] PS(619447,ffff42fdefa0,python3.9):2025-04-03-09:35:53.892.122 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619447,ffff08bd8fa0,python3.9):2025-04-03-09:35:53.892.195 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(619447,fffed57aefa0,python3.9):2025-04-03-09:35:53.892.333 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.413 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.436 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.453 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.476 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.495 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.511 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.530 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.643 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.673 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.721 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.753 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.770 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.793 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.811 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.845 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.864 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.882 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.900 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.935 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.892.954 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.291 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.313 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.344 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.364 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.381 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.400 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619447,ffffb243e020,python3.9):2025-04-03-09:35:53.893.655 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:53.923.075 [awatchdog_monitor.c:160](tid:619479) thread 619706 not exist [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.964.757 [runtime.cc:5239] 619596 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:53.964.803 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:53.964.817 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619447,python3.9):2025-04-03-09:35:53.964.831 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619596] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.964.857 [runtime.cc:3077] 619596 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=7, devId=7, tdt status=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:53.964.932 [npu_driver.cc:551] 619596 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:54.328.173 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619447,ffffb243e020,python3.9):2025-04-03-09:35:54.328.306 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.679.965 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.346 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.383 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.460 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.479 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.955 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.976 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.680.991 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.005 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.029 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.046 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.680 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.704 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.720 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.735 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.904 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.924 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.946 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.681.963 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.032 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.050 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.236 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.255 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.372 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.391 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.505 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.523 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.536 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.550 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.564 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.578 [heartbeat.cc:145] [619447][DeInit]heartbeat deinit begin [INFO] HCCL(619447,python3.9):2025-04-03-09:35:54.682.592 [heartbeat.cc:183] [619447][DeInit]heartbeat deinit end [INFO] DEVICE(619447,ffffb243e020,python3.9):2025-04-03-09:35:54.682.904 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 7, group name: hccl_world_group [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:54.682.962 [hash_data.cpp:63] >>> (tid:619447) HashData uninitialize success [INFO] PROFILING(619447,python3.9):2025-04-03-09:35:54.682.988 [msprof_reporter_mgr.cpp:270] >>> (tid:619447) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.684.794 [api_impl.cc:2003] 619447 DevFree: device free mem=0x12c041200000 [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:54.686.014 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619447,ffffb243e020,python3.9):2025-04-03-09:35:54.686.049 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:54.686.187 [adx_dump_record.cpp:229][tid:619447] start to dump uninit [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:54.686.219 [adx_dump_record.cpp:239][tid:619447] dump uninit success [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:54.686.235 [adx_dump_record.cpp:674][tid:619673] exit record file thread [INFO] GE(619447,python3.9):2025-04-03-09:35:54.750.886 [operator_factory_impl.cc:392]619447 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.174 [operator_factory_impl.cc:392]619447 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.232 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.251 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.262 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.273 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.283 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.296 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.308 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.766.322 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.914.334 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.914.540 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] GE(619447,python3.9):2025-04-03-09:35:54.914.593 [opsproto_manager.cc:66]619447 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.970.274 [api_impl.cc:7151] 619447 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.970.478 [api_impl.cc:7135] 619447 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.972.638 [api_impl.cc:7151] 619447 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.972.806 [api_impl.cc:7135] 619447 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.974.512 [api_impl.cc:7121] 619447 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.978.134 [api_impl.cc:7121] 619447 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:54.984.703 [osal.cc:321] 619447 ThreadExit: ThreadExit start. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:55.176.751 [stars_engine.cc:1833] 619596 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:55.176.826 [stars_engine.cc:1836] 619596 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619447,ffffb243e020,python3.9):2025-04-03-09:35:55.177.121 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619447,python3.9):2025-04-03-09:35:55.209.524 [ops_kernel_builder_registry.cc:71]619447 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619447,python3.9):2025-04-03-09:35:55.209.609 [ops_kernel_builder_registry.cc:41]619447 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:56.516.198 [adx_server_manager.cpp:236][tid:619674] server manager stop [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:56.594.647 [adx_hdc_epoll.cpp:57][tid:619447] start to close hdc epoll [INFO] GE(619447,python3.9):2025-04-03-09:35:56.696.310 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab12635cd0]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.698.262 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab1263a300]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.700.991 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab12664780]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.704.217 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab12637310]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.705.141 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab13a70900]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.706.051 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab13a988f0]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.707.304 [op_impl_registry_holder_manager.cc:31]619447 CloseHandle:start close handle, handle[0xaaab13b5bf70]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.726.877 [mem_manager.cc:118]619447 Finalize:MemManager Finalize. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.726.921 [session_scope_mem_allocator.cc:73]619447 FreeAllMemory:Free all memory [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.284 [session_scope_mem_allocator.cc:73]619447 FreeAllMemory:Free all memory [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.318 [caching_allocator.cc:474]619447 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.350 [caching_allocator.cc:391]619447 FreeBlocks:Free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.369 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.381 [caching_allocator.cc:422]619447 FreeBlockBins:Free block bins. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.394 [caching_allocator.cc:474]619447 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.409 [caching_allocator.cc:391]619447 FreeBlocks:Free blocks. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.422 [caching_allocator.cc:358]619447 FreeCachedBlocks:Free cached blocks [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.435 [caching_allocator.cc:422]619447 FreeBlockBins:Free block bins. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.455 [mem_manager.cc:145]619447 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.476 [rts_caching_mem_allocator.cc:150]619447 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619447,python3.9):2025-04-03-09:35:56.729.540 [rts_caching_mem_allocator.cc:136]619447 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.842.898 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.842.969 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.009 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.021 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.031 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.040 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.053 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.066 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.078 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.091 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.103 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.115 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.127 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.139 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.152 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] HCCL(619447,python3.9):2025-04-03-09:35:56.843.164 [network_manager.cc:875] [619447]destroy all nic/vnic. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.850.421 [runtime.cc:1570] 619447 ~Runtime: deconstruct runtime [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.850.455 [runtime.cc:1577] 619447 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.408 [raw_device.cc:262] 619447 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.465 [engine.cc:188] 619447 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.491 [ringbuffer_maintain_task.cc:83] 619447 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.509 [stream.cc:1400] 619447 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.523 [stream.cc:2595] 619447 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=7 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.547 [npu_driver.cc:751] 619447 SqTaskSend: device_id=7, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.564 [stars_engine.cc:893] 619447 SendTask: Sq task send finished, device_id=7, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.581 [stream.cc:1494] 619447 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.604 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.623 [stream.cc:2886] 619447 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.638 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.648 [stream.cc:2886] 619447 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.660 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.673 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.708 [task.cc:185] 619447 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.723 [logger.cc:152] 619447 TaskFinished: device_id=7, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.746 [stars_engine.cc:437] 619447 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.764 [stream.cc:2886] 619447 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.778 [stars_engine.cc:524] 619447 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.792 [npu_driver.cc:5547] 619447 GetSqHead: deviceId=7, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.813 [stream.cc:852] 619447 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.832 [stream.cc:938] 619447 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.851.846 [stream.cc:1507] 619447 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.274 [stream_sqcq_manage.cc:451] 619447 FreeLogicCq: Return(0), threadIdentifier(281473672536096), devId(7), tsId(0), cqId(339), isFastCq(0). [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.322 [npu_driver.cc:6163] 619447 StreamUnBindLogicCq: success: deviceId=7, tsId=0, streamId=0, logicCq=339. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.377 [npu_driver.cc:1180] 619447 LogicCqFree: Succ:device_id=7,ts_id=0,type=2,logic_cq_id=339, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.397 [stream.cc:237] 619447 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.412 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.855.426 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.032 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.055 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.072 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.095 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=0, device_id=7, tsId=0, streamFlag=192 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.252 [stars_engine.cc:206] 619597 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.385 [stars_engine.cc:191] 619447 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.406 [osal.cc:225] 619447 Join: mmJoinTask failed, thread=281472026234784 does not exist! [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.419 [stars_engine.cc:150] 619447 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.441 [runtime.cc:5239] 619447 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=7, deviceId=7. [INFO] TDT(619447,python3.9):2025-04-03-09:35:56.856.461 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619447] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.477 [runtime.cc:3003] 619447 StopAicpuExecutor: TsdClose success, userDeviceId=7, devId=7, tdt status=0 [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:56.856.512 [atrace_api.c:118](tid:619447) AtraceDestroy start. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:56.856.555 [tracer_mgr_operate.c:248](tid:619447) destroy object RUNTIME_ATRACE_DEV7_TS0, exitSave(false). [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:56.856.577 [atrace_api.c:120](tid:619447) AtraceDestroy end. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.596 [stream.cc:237] 619447 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.613 [stream_sqcq_manage.cc:216] 619447 DeAllocStreamSqCq: deviceId=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.626 [npu_driver.cc:966] 619447 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.839 [npu_driver.cc:974] 619447 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=7, tsId=0, drvFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.860 [stream_sqcq_manage.cc:228] 619447 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.874 [npu_driver.cc:432] 619447 StreamIdFree: stream id free begin, device_id=7, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.856.890 [npu_driver.cc:441] 619447 StreamIdFree: Stream id free success, stream_id=1, device_id=7, tsId=0, streamFlag=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.867.504 [stars_engine.cc:58] 619447 ~StarsEngine: Destructor. [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.867.533 [engine.cc:84] 619447 ~Engine: Destructor. [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.342 [trs_dev_drv.c:370][ascend][curpid:619447,619447][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=7; fd=36; mmapFd=37) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.371 [devmm_svm.c:2238][ascend][curpid:619447,619447][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=7) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.387 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.407 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.427 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.446 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.464 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev7 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.486 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev7 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.507 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev7 Cached_size:1961984Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.520 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev7 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.554 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev7 Cached_size:2097152Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.568 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev7 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.586 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev7 Cached_size:2097152Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.605 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev7 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.625 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev7 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.643 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev7 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.663 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev7 Cached_size:23068672Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.675 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev7 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.692 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev7 Cached_size:6291456Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.708 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev7 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.726 [svm_mem_statistics.c:140][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev7 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.868.747 [svm_mem_statistics.c:146][ascend][curpid:619447,619447][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev7 Cached_size:16777216Bytes [INFO] DRV(619447,python3.9):2025-04-03-09:35:56.937.473 [drv_log_user.c:623][ascend][curpid:619447,619447][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=7; phyid=7; vfid=0; hostpid=619447; devpid=30963) [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.937.517 [npu_driver.cc:3045] 619447 DeviceClose: Close device success, device_id=7, tsId=0 [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:56.937.532 [raw_device.cc:125] 619447 ~Device: deconstruct device [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:57.046.265 [plugin_old_arch.cc:25] 619447 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:57.046.300 [runtime_keeper.cc:167] 619447 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619447,python3.9):2025-04-03-09:35:57.047.538 [task_fail_callback_data_manager.cc:63] 619447 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:57.152.722 [stacktrace_signal.c:188](tid:619447) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:57.209.808 [atrace_client_communication.c:141](tid:619447) send [end] msg. [INFO] DRV(619447,python3.9):2025-04-03-09:35:57.211.402 [hdc_core.c:1982][ascend][curpid:619447,619447][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4281; pid=619447) [INFO] IDEDD(619447,python3.9):2025-04-03-09:35:57.211.434 [hdc_api.cpp:671][tid:619447] connect succ, peer_node: 0, peer_devid: 7 [INFO] DRV(619447,python3.9):2025-04-03-09:35:57.212.543 [hdc_client.c:417][ascend][curpid:619447,619447][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4281) [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:57.212.612 [atrace_client_communication.c:164](tid:619447) send [end] msg success. [INFO] DRV(619447,python3.9):2025-04-03-09:35:58.282.235 [hdc_client.c:417][ascend][curpid:619447,619579][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4242) [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:58.282.282 [atrace_client_core.c:127](tid:619579) atrace receive thread exited, devId=7. [INFO] ATRACE(619447,python3.9):2025-04-03-09:35:58.282.906 [awatchdog_monitor.c:234](tid:619447) AwdMonitorExit [INFO] DRV(619447,python3.9):2025-04-03-09:35:58.285.515 [uda_user.c:147][ascend][curpid:619447,619447][drv][devmng][uda_char_dev_open]The fd and pid info: (fd=3; pid=619447) [INFO] DRV(619447,python3.9):2025-04-03-09:35:58.285.683 [uda_user.c:317][ascend][curpid:619447,619447][drv][devmng][_uda_init]User info. (admin_flag=1; local_flag=0; max_dev_num=100; max_udev_num=1124; uda_dev_num_davinci=8; uda_dev_num_kunpeng=0) output_allreduce/scheduler.log0000644000175100017500000026343714773362773017355 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619417,ffff9f32c020,python3.9):2025-04-03-09:35:26.640.652 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.070 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.151 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.194 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.210 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.224 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.239 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.252 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.263 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.275 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.286 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.295 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.307 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.319 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.328 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.339 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.350 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.359 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.370 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.381 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.390 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.402 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.413 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.443 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.456 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.469 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.478 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.496 [ir_data_type_symbol_store.cc:240]619417 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.512 [ir_data_type_symbol_store.cc:240]619417 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.523 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.532 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.542 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.554 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.564 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.573 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.583 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.592 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.602 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.613 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.624 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.634 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.645 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.656 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.665 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.675 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.688 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.698 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.710 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.719 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.729 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.738 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.747 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.757 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.768 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.777 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.789 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.800 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.810 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.819 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.829 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.838 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.847 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.857 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.867 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.876 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.886 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.897 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.906 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.917 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.926 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.941 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.952 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.962 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.971 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.980 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.056.992 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.001 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.010 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.020 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.029 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.038 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.049 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.060 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.069 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.081 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.090 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.098 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.108 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.118 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.127 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.135 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.151 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.163 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.172 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.180 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.190 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.205 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.214 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.225 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.234 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.244 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.255 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.263 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.273 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.283 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.292 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.300 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.310 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.320 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.329 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.339 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.348 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.365 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.375 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.383 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.394 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.403 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.413 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.422 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.430 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.440 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.454 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.465 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.476 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.484 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.494 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.505 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.514 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.523 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.531 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.540 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.552 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.562 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.571 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.581 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.590 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.600 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.611 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.621 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.631 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.641 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.651 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.660 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.670 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.680 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.689 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.700 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.712 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.721 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.731 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.740 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.752 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.762 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.772 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.784 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.793 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.803 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.812 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.827 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.842 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.852 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.863 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.873 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.882 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.892 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.901 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.910 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.922 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.932 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.943 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.952 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.964 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.974 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.984 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.057.994 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.003 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.014 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.024 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.037 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.048 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.059 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.068 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.077 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.091 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.112 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.124 [ir_data_type_symbol_store.cc:173]619417 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.133 [ir_data_type_symbol_store.cc:219]619417 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619417,python3.9):2025-04-03-09:35:27.058.149 [ir_data_type_symbol_store.cc:182]619417 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.004 [awatchdog_core.c:99](tid:619417) watchdog get platform host side [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.152 [awatchdog_monitor.c:127](tid:619417) create monitor thread successfully, tid :281472715419552 [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.183 [awatchdog_monitor.c:226](tid:619417) AwdMonitorInit [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.354 [trace_attr.c:99](tid:619417) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.368 [trace_attr.c:189](tid:619417) attr init success, timeout=0ms. [WARNING] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.378 [trace_system_api.c:115](tid:619417) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.446 [trace_recorder.c:40](tid:619417) home_directory=/home/jenkins. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.457 [trace_recorder.c:121](tid:619417) use root path: /home/jenkins/ascend [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.466 [trace_event.c:151](tid:619417) init trace event success [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.064.503 [atrace_client_core.c:185](tid:619417) atrace client init success. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.609 [stacktrace_dumper.c:218](tid:619417) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.624 [trace_system_api.c:115](tid:619417) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.721 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 2 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.733 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 15 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.742 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 3 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.751 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 4 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.759 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 5 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.768 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 6 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.776 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 7 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.784 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 8 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.792 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 11 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.800 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 24 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.808 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 25 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.817 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 31 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.825 [stacktrace_signal.c:165](tid:619417) register signal handler for signo 35 succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.832 [stacktrace_signal.c:168](tid:619417) trace signal init succeed. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:27.065.839 [trace_core.c:51](tid:619417) trace init success. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.066.416 [task_fail_callback_data_manager.cc:58] 619417 TaskFailCallBackManager: Constructor. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.074.902 [operator_factory_impl.cc:232]619417 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619417,python3.9):2025-04-03-09:35:27.074.924 [operator_factory_impl.cc:246]619417 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619417,python3.9):2025-04-03-09:35:27.074.934 [operator_factory_impl.cc:239]619417 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619417,python3.9):2025-04-03-09:35:27.081.399 [adapter_rts.cc:2720] [619417][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.097.386 [op_impl_registry.cc:122]619417 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.097.431 [op_impl_registry.cc:122]619417 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619417,python3.9):2025-04-03-09:35:27.097.462 [op_impl_registry.cc:122]619417 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619417,python3.9):2025-04-03-09:35:27.108.019 [prof_cann_plugin.cpp:256] (tid:619417) Register module[6] callback. [INFO] OP(619417,python3.9):2025-04-03-09:35:27.161.644 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619417] cachelimit is 10000 [INFO] OP(619417,python3.9):2025-04-03-09:35:27.165.867 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619417] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619417,python3.9):2025-04-03-09:35:27.165.917 [op_dfx.cpp:54][NNOP][SystemConfig][619417] enable debug kernel flag 0. [INFO] PROFILING(619417,python3.9):2025-04-03-09:35:27.165.930 [prof_cann_plugin.cpp:256] (tid:619417) Register module[61] callback. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.214.546 [mem_manager.cc:138]619417 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.220.840 [cache_policy.cc:47]619417 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619417,python3.9):2025-04-03-09:35:27.233.823 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.233.940 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.035 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.056 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.076 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.095 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.111 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.128 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.147 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.164 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.179 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.194 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.211 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.227 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.244 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.262 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.277 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.293 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.309 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.326 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.413 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.500 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.534 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.627 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.646 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.666 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.682 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.698 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.712 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.728 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.744 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.763 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.780 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.795 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.810 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.825 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.841 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.857 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.873 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.890 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.909 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.234.926 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.007 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.093 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.179 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.266 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.354 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.373 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.464 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.551 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.571 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.655 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.737 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.765 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.781 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.801 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.816 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.832 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.848 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.867 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.884 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.900 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.918 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.936 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.953 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.235.969 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.055 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.144 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.240 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.325 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.346 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.364 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.381 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.467 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.559 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.578 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.665 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.685 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.768 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.855 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.236.942 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.024 [addn.cc:29]619417 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.311 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.405 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.486 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.569 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.645 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.722 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.805 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.883 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.894 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619417,python3.9):2025-04-03-09:35:27.237.974 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619417,python3.9):2025-04-03-09:35:27.238.055 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619417,python3.9):2025-04-03-09:35:27.238.134 [symbolic_kernel_factory.cc:27]619417 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619417,python3.9):2025-04-03-09:35:27.244.069 [process_node_engine_manager.cc:140]619417 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619417,python3.9):2025-04-03-09:35:27.244.186 [process_node_engine_manager.cc:140]619417 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619417,python3.9):2025-04-03-09:35:27.250.142 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619417,python3.9):2025-04-03-09:35:27.250.170 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619417,python3.9):2025-04-03-09:35:27.250.661 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619417] start packet count timer [INFO] PROFILING(619417,python3.9):2025-04-03-09:35:27.271.708 [prof_atls_plugin.cpp:117] (tid:619417) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619417,python3.9):2025-04-03-09:35:27.271.741 [prof_atls_plugin.cpp:117] (tid:619417) RegisterProfileCallback, callback type is 8 [INFO] ME(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.332.794 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.333.222 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.368.462 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.368.583 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.369.314 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.370.819 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.370.881 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.370.907 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.370.995 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.371.020 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.371.062 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.371.081 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.372.800 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.372.823 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.372.848 [acl.cpp:753]619417 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.372.899 [runtime_keeper.cc:78] 619417 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.215 [plugin_old_arch.cc:16] 619417 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaaf4944f80 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.240 [runtime_keeper.cc:149] 619417 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.250 [runtime_keeper.cc:221] 619417 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.280 [runtime.cc:1387] 619417 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.304 [runtime.cc:1163] 619417 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.322 [npu_driver.cc:4062] 619417 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.336 [runtime.cc:5096] 619417 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.345 [runtime.cc:1173] 619417 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.354 [runtime.cc:1183] 619417 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.366 [runtime.cc:1193] 619417 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.374 [profiler.cc:84] 619417 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.386 [runtime.cc:1209] 619417 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.394 [runtime.cc:1219] 619417 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.403 [runtime.cc:1230] 619417 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.419 [runtime.cc:1242] 619417 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.428 [runtime.cc:1264] 619417 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.437 [runtime.cc:1270] 619417 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.537 [runtime.cc:109] 619417 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.919 [runtime.cc:121] 619417 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.985 [runtime.cc:1314] 619417 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.376.996 [runtime.cc:1322] 619417 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.123 [runtime.cc:1338] 619417 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.178 [utils.cc:105] 619417 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.224 [runtime.cc:1664] 619417 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.253 [runtime.cc:1688] 619417 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.268 [runtime.cc:1953] 619417 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.290 [runtime.cc:1477] 619417 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619417,python3.9):2025-04-03-09:35:27.377.304 [prof_atls_plugin.cpp:210] (tid:619417) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.316 [heterogenous.cc:19] 619417 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:27.377.343 [api_c_soc.cc:70] 619417 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.377.353 [acl.cpp:115]619417 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.377.363 [acl.cpp:760]619417 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.377.395 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.405.722 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.436.387 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.436.803 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.437.215 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.437.977 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.438.370 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.439.947 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.439.994 [acl.cpp:753]619417 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.440.015 [acl.cpp:115]619417 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619417,python3.9):2025-04-03-09:35:27.440.025 [acl.cpp:760]619417 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:27.440.045 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.388.497 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.390.206 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.422.503 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.431.301 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.434.247 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.437.239 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.465.242 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619417,python3.9):2025-04-03-09:35:28.480.036 [platform_info.cpp:853]619417 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.402.569 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.404.329 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.404.450 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.406.669 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.406.790 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.406.896 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.406.996 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.407.118 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.408.227 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.408.353 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.248 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.368 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.472 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.572 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.665 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619417,python3.9):2025-04-03-09:35:29.410.760 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619417,python3.9):2025-04-03-09:35:30.035.607 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619417,python3.9):2025-04-03-09:35:34.300.969 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.620.960 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.621.036 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.621.068 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619417,python3.9):2025-04-03-09:35:34.666.958 [ops_kernel_builder_registry.cc:55]619417 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619417,python3.9):2025-04-03-09:35:34.667.057 [ops_kernel_builder_registry.cc:28]619417 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.705.875 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.705.940 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.705.991 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.706.006 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619417,ffff9f32c020,python3.9):2025-04-03-09:35:34.706.019 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619417,python3.9):2025-04-03-09:35:35.580.163 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619417:281473352646688,MainProcess):2025-04-03-09:35:37.437.680 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619417:281473352646688,MainProcess):2025-04-03-09:35:37.438.261 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619417:281473352646688,MainProcess):2025-04-03-09:35:37.449.111 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] PS(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.568 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_SCHED [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.645 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.661 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.687 [mindspore/ccsrc/distributed/rpc/tcp/tcp_server.cc:48] InitializeImpl] Config cluster threads num : . [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.855 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471740702624 [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.536.967 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471732248480 [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.537.021 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:232] StartServerSocket] Start server succ, fd: 16, url: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:37.537.121 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619417,ffff3e1c6fa0,python3.9):2025-04-03-09:35:37.537.185 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:489] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (0/8) [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:38.037.200 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [INFO] DISTRIBUTED(619417,ffff3e1c6fa0,python3.9):2025-04-03-09:35:38.037.287 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:489] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (0/8) [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:38.537.278 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [INFO] DISTRIBUTED(619417,ffff3e1c6fa0,python3.9):2025-04-03-09:35:38.537.378 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:489] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (0/8) [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:38.582.777 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 1(role: MS_WORKER), rank id: 1, device id: 1, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 1, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:38.587.814 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 6(role: MS_WORKER), rank id: 6, device id: 6, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 2, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:38.596.846 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 7(role: MS_WORKER), rank id: 7, device id: 7, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 3, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:38.955.500 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 5(role: MS_WORKER), rank id: 5, device id: 5, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 4, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.002.057 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 2(role: MS_WORKER), rank id: 2, device id: 2, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 5, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:39.037.351 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(4/1200). [INFO] DISTRIBUTED(619417,ffff3e1c6fa0,python3.9):2025-04-03-09:35:39.037.455 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:489] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (5/8) [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:39.537.422 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(5/1200). [INFO] DISTRIBUTED(619417,ffff3e1c6fa0,python3.9):2025-04-03-09:35:39.537.535 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:489] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (5/8) [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.647.186 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 4(role: MS_WORKER), rank id: 4, device id: 4, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 6, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.844.868 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 0(role: MS_WORKER), rank id: 0, device id: 0, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 7, expected node number: 8 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.959.782 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:772] ReassignNodeRank] Rank ids are already set by numeric node ids. No need to reassign them. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.959.821 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:549] TransitionToInitialized] The cluster topology has been constructed successfully. [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:39.959.838 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:254] ProcessRegister] The new node: 3(role: MS_WORKER), rank id: 3, device id: 3, hostname: devserver-c233-2, ip: 127.0.0.1 is registered successfully. Currently registered node number: 8, expected node number: 8 [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.511 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 2500.86 msec. [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.528 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.542 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0 msec. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.563 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 2501.11 msec. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.576 [mindspore/ccsrc/distributed/init.cc:158] InitializeCollective] Scheduler node does not need to initialize collective communication. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.590 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 0.012 msec. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.601 [mindspore/ccsrc/distributed/init.cc:68] Initialize] Scheduler starts to wait for cluster to exit. [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.615 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:98] Finalize] The meta server node can not be finalized because there are still 8 alive nodes. [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:40.037.626 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:154] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:45.037.727 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:98] Finalize] The meta server node can not be finalized because there are still 8 alive nodes. [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:45.037.754 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:154] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:50.037.867 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:98] Finalize] The meta server node can not be finalized because there are still 8 alive nodes. [WARNING] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:50.037.908 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:154] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:50.850.134 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 0 has unregistered. [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:50.850.516 [mindspore/ccsrc/distributed/rpc/tcp/connection.cc:79] SocketEventHandler] Event value fd: 28, events: 8193, state: 4, errcode: 11, errno: 11 Resource temporarily unavailable, remote peer: 127.0.0.1:58788, type:1, remote: 1, count: 1, this peer: 127.0.0.1:8118, please check remote peer address: 127.0.0.1:58788 in worker log to find out which worker disconnected. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:50.900.661 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 28 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58788 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:50.900.785 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 31 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:46860 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.007.024 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 2 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.007.245 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 26 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58774 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.007.659 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 21 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58742 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.087.922 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 1 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.088.105 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 22 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58744 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.088.583 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 17 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58712 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.092.537 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 6 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.092.730 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 23 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58750 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.093.167 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 18 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58720 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.101.225 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 7 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.101.408 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 24 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58756 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.101.808 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 19 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58722 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.151.315 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 4 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.151.499 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 27 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58776 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.151.924 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 30 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:46846 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.460.718 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 5 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.460.896 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 25 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58764 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.461.268 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 20 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:58728 [WARNING] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.464.416 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:323] ProcessUnregister] Node 3 has unregistered. [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.464.609 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 29 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:46832 [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:51.465.112 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 32 state is kDisconnecting. Start disconnecting from 127.0.0.1:8118 to 127.0.0.1:46870 [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.038.036 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619417,ffff3e9d6fa0,python3.9):2025-04-03-09:35:55.038.152 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.038.314 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.038.332 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619417,ffff3f1e6fa0,python3.9):2025-04-03-09:35:55.038.455 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.038.682 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.038.723 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619417,ffff9f32c020,python3.9):2025-04-03-09:35:55.039.616 [mindspore/ccsrc/distributed/init.cc:74] Initialize] Scheduler ends waiting for cluster to exit. [INFO] GE(619417,python3.9):2025-04-03-09:35:55.079.065 [ops_kernel_builder_registry.cc:71]619417 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619417,python3.9):2025-04-03-09:35:55.079.131 [ops_kernel_builder_registry.cc:41]619417 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] GE(619417,python3.9):2025-04-03-09:35:55.512.019 [mem_manager.cc:118]619417 Finalize:MemManager Finalize. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.850 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.901 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.911 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.920 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.928 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.936 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.944 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.952 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.960 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.577.993 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.001 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.009 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.016 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.024 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.031 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] HCCL(619417,python3.9):2025-04-03-09:35:55.578.038 [network_manager.cc:875] [619417]destroy all nic/vnic. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:55.580.763 [runtime.cc:1570] 619417 ~Runtime: deconstruct runtime [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:55.580.790 [runtime.cc:1577] 619417 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:55.699.098 [plugin_old_arch.cc:25] 619417 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:55.699.137 [runtime_keeper.cc:167] 619417 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619417,python3.9):2025-04-03-09:35:55.699.929 [task_fail_callback_data_manager.cc:63] 619417 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:55.796.566 [stacktrace_signal.c:188](tid:619417) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619417,python3.9):2025-04-03-09:35:55.882.559 [awatchdog_monitor.c:234](tid:619417) AwdMonitorExit output_allreduce/worker_0.log0000644000175100017500000714126714773362776017135 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619419,ffff921ed020,python3.9):2025-04-03-09:35:26.657.595 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:26.671.176 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.396 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.456 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.484 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.502 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.517 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.532 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.547 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.559 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.574 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.589 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.600 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.614 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.627 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.638 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.651 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.666 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.677 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.690 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.704 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.715 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.727 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.748 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.759 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.775 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.788 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.798 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.817 [ir_data_type_symbol_store.cc:240]619419 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.831 [ir_data_type_symbol_store.cc:240]619419 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.844 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.856 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.869 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.884 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.895 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.907 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.919 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.930 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.942 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.954 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.965 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.976 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.064.991 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.003 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.014 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.031 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.043 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.055 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.067 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.077 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.089 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.103 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.114 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.125 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.137 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.148 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.160 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.171 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.184 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.196 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.209 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.221 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.232 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.244 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.256 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.267 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.278 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.292 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.304 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.315 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.331 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.343 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.355 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.367 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.377 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.390 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.403 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.415 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.426 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.438 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.449 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.460 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.472 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.483 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.494 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.508 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.520 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.530 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.542 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.554 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.565 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.576 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.587 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.601 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.612 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.624 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.640 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.652 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.664 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.674 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.686 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.697 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.707 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.717 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.729 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.742 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.753 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.764 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.776 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.788 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.799 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.809 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.820 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.832 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.843 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.854 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.869 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.882 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.894 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.905 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.915 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.932 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.943 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.955 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.966 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.977 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.065.989 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.001 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.012 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.023 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.033 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.044 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.060 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.071 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.082 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.093 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.104 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.118 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.129 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.140 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.152 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.163 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.175 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.186 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.196 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.208 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.219 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.237 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.248 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.259 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.271 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.282 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.293 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.304 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.315 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.326 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.337 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.350 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.362 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.379 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.394 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.405 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.418 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.432 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.443 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.454 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.465 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.476 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.488 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.501 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.512 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.526 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.537 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.548 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.561 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.574 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.584 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.597 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.609 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.620 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.633 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.647 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.658 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.668 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.685 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.712 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.723 [ir_data_type_symbol_store.cc:173]619419 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.734 [ir_data_type_symbol_store.cc:219]619419 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619419,python3.9):2025-04-03-09:35:27.066.750 [ir_data_type_symbol_store.cc:182]619419 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.595 [awatchdog_core.c:99](tid:619419) watchdog get platform host side [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.711 [awatchdog_monitor.c:127](tid:619419) create monitor thread successfully, tid :281472496009120 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.727 [awatchdog_monitor.c:226](tid:619419) AwdMonitorInit [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.893 [trace_attr.c:99](tid:619419) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.917 [trace_attr.c:189](tid:619419) attr init success, timeout=0ms. [WARNING] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.930 [trace_system_api.c:115](tid:619419) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.981 [trace_recorder.c:40](tid:619419) home_directory=/home/jenkins. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.072.994 [trace_recorder.c:121](tid:619419) use root path: /home/jenkins/ascend [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.073.004 [trace_event.c:151](tid:619419) init trace event success [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.073.043 [atrace_client_core.c:185](tid:619419) atrace client init success. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.126 [stacktrace_dumper.c:218](tid:619419) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.144 [trace_system_api.c:115](tid:619419) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.225 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 2 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.239 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 15 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.249 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 3 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.260 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 4 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.270 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 5 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.280 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 6 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.290 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 7 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.300 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 8 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.309 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 11 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.319 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 24 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.329 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 25 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.338 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 31 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.347 [stacktrace_signal.c:165](tid:619419) register signal handler for signo 35 succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.357 [stacktrace_signal.c:168](tid:619419) trace signal init succeed. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:27.074.365 [trace_core.c:51](tid:619419) trace init success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.074.948 [task_fail_callback_data_manager.cc:58] 619419 TaskFailCallBackManager: Constructor. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.083.290 [operator_factory_impl.cc:232]619419 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619419,python3.9):2025-04-03-09:35:27.083.322 [operator_factory_impl.cc:246]619419 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619419,python3.9):2025-04-03-09:35:27.083.332 [operator_factory_impl.cc:239]619419 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619419,python3.9):2025-04-03-09:35:27.089.642 [adapter_rts.cc:2720] [619419][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.103.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.103.915 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:27.103.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:27.113.354 [prof_cann_plugin.cpp:256] (tid:619419) Register module[6] callback. [INFO] OP(619419,python3.9):2025-04-03-09:35:27.167.070 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619419] cachelimit is 10000 [INFO] OP(619419,python3.9):2025-04-03-09:35:27.170.761 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619419] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619419,python3.9):2025-04-03-09:35:27.170.804 [op_dfx.cpp:54][NNOP][SystemConfig][619419] enable debug kernel flag 0. [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:27.170.819 [prof_cann_plugin.cpp:256] (tid:619419) Register module[61] callback. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.216.492 [mem_manager.cc:138]619419 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.222.516 [cache_policy.cc:47]619419 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.025 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.135 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.225 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.249 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.269 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.291 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.312 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.330 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.349 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.372 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.392 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.415 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.433 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.451 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.470 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.487 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.505 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.525 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.545 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.563 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.652 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.749 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.772 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.862 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.885 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.904 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.924 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.941 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.960 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.978 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.235.996 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.017 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.037 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.059 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.076 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.093 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.111 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.129 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.146 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.178 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.197 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.215 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.300 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.385 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.472 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.560 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.649 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.672 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.762 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.854 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.877 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.236.957 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.048 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.070 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.088 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.107 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.125 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.147 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.165 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.182 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.200 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.219 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.237 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.254 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.273 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.293 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.387 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.478 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.562 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.645 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.667 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.685 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.704 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.787 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.876 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.897 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.237.985 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.008 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.090 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.182 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.271 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.354 [addn.cc:29]619419 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.644 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.727 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.810 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.890 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619419,python3.9):2025-04-03-09:35:27.238.967 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.049 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.126 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.205 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.218 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.296 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.376 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619419,python3.9):2025-04-03-09:35:27.239.452 [symbolic_kernel_factory.cc:27]619419 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619419,python3.9):2025-04-03-09:35:27.245.171 [process_node_engine_manager.cc:140]619419 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619419,python3.9):2025-04-03-09:35:27.245.276 [process_node_engine_manager.cc:140]619419 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619419,python3.9):2025-04-03-09:35:27.250.642 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619419,python3.9):2025-04-03-09:35:27.250.669 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619419,python3.9):2025-04-03-09:35:27.251.175 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619419] start packet count timer [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:27.271.708 [prof_atls_plugin.cpp:117] (tid:619419) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:27.271.749 [prof_atls_plugin.cpp:117] (tid:619419) RegisterProfileCallback, callback type is 8 [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.332.794 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.333.222 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.368.462 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.368.570 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 0 [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.368.600 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.369.302 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.370.819 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.370.881 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.370.910 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.370.995 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.371.020 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.371.062 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.371.082 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.372.798 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.372.827 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.372.852 [acl.cpp:753]619419 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.372.899 [runtime_keeper.cc:78] 619419 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.214 [plugin_old_arch.cc:16] 619419 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaab5644ab0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.246 [runtime_keeper.cc:149] 619419 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.256 [runtime_keeper.cc:221] 619419 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.283 [runtime.cc:1387] 619419 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.306 [runtime.cc:1163] 619419 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.322 [npu_driver.cc:4062] 619419 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.336 [runtime.cc:5096] 619419 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.347 [runtime.cc:1173] 619419 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.357 [runtime.cc:1183] 619419 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.369 [runtime.cc:1193] 619419 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.379 [profiler.cc:84] 619419 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.393 [runtime.cc:1209] 619419 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.413 [runtime.cc:1219] 619419 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.423 [runtime.cc:1230] 619419 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.432 [runtime.cc:1242] 619419 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.442 [runtime.cc:1264] 619419 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.452 [runtime.cc:1270] 619419 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.558 [runtime.cc:109] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.376.939 [runtime.cc:121] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.008 [runtime.cc:1314] 619419 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.021 [runtime.cc:1322] 619419 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.134 [runtime.cc:1338] 619419 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.178 [utils.cc:105] 619419 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.224 [runtime.cc:1664] 619419 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.254 [runtime.cc:1688] 619419 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.268 [runtime.cc:1953] 619419 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.291 [runtime.cc:1477] 619419 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:27.377.306 [prof_atls_plugin.cpp:210] (tid:619419) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.320 [heterogenous.cc:19] 619419 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:27.377.342 [api_c_soc.cc:70] 619419 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.377.355 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.377.365 [acl.cpp:760]619419 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.377.395 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.405.722 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.436.930 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.437.363 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.437.786 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.438.555 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.438.971 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.440.542 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.440.579 [acl.cpp:753]619419 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.440.600 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:27.440.614 [acl.cpp:760]619419 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:27.440.633 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.323.778 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.325.557 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.345.114 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.353.790 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.356.718 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.359.684 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.387.359 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619419,python3.9):2025-04-03-09:35:28.401.845 [platform_info.cpp:853]619419 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.413.792 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.415.626 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.415.760 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.418.000 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.418.138 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.418.255 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.418.393 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.418.501 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.419.625 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.419.760 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.421.681 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.421.813 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.421.929 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.422.040 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.422.145 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619419,python3.9):2025-04-03-09:35:29.422.251 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619419,python3.9):2025-04-03-09:35:30.043.729 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619419,python3.9):2025-04-03-09:35:34.272.502 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.543.073 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.543.179 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.543.216 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619419,python3.9):2025-04-03-09:35:34.591.741 [ops_kernel_builder_registry.cc:55]619419 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619419,python3.9):2025-04-03-09:35:34.591.878 [ops_kernel_builder_registry.cc:28]619419 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.631.176 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.631.297 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.631.423 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.631.447 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:34.631.463 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619419,python3.9):2025-04-03-09:35:35.468.064 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:37.237.980 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:37.238.675 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:37.250.021 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.341.036 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:37.341.125 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.342.403 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.342.460 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.342.482 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.342.515 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.342.812 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471521292192 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.009 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471512838048 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.037 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.206 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471504383904 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.374 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471495929760 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.404 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.426 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.640 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58694, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.343.686 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [INFO] DISTRIBUTED(619419,ffff31087fa0,python3.9):2025-04-03-09:35:37.343.757 [mindspore/ccsrc/distributed/rpc/tcp/connection.cc:239] ReconnectSourceSocket] Connection from 127.0.0.1:58694 to 127.0.0.1:8118 is not available yet, errno: 0 Success, events: 8220 [INFO] DISTRIBUTED(619419,ffff31087fa0,python3.9):2025-04-03-09:35:37.343.827 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:35] DoDisconnect] Failed to call connect, fd: 20from : 127.0.0.1:58694 to: 127.0.0.1:8118, events: 8216, errno: 111 Connection refused [INFO] DISTRIBUTED(619419,ffff31087fa0,python3.9):2025-04-03-09:35:37.843.799 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:281] EventCallBack] The connection fd : 20 state is kDisconnecting. Start disconnecting from 127.0.0.1:58694 to 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:37.843.813 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:76] Connect] Failed to connect to the tcp server : 127.0.0.1:8118, retry to reconnect(1/1)... [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.343.953 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:173] Register] Failed to connect to the meta server node url: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.343.987 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:363] ReconnectWithTimeoutWindow] Failed to register and try to reconnect to the meta server. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.844.078 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:386] Reconnect] Start reconnecting business client. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.844.107 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.844.244 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58788, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:38.844.269 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619419,ffff320a7fa0,python3.9):2025-04-03-09:35:38.844.326 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58788 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.361 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.391 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.410 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:393] Reconnect] Start reconnecting heartbeat client. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.439 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.560 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 22 source: 127.0.0.1:46860, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.344.581 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 3 [WARNING] DISTRIBUTED(619419,ffff31087fa0,python3.9):2025-04-03-09:35:39.344.582 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:46860 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.844.671 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.844.700 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.844.721 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.844.739 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.845.031 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 0 has been registered successfully. [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:39.845.309 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619419,ffff23ffefa0,python3.9):2025-04-03-09:35:39.845.371 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619419,ffff23ffefa0,python3.9):2025-04-03-09:35:39.845.410 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.345.394 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.505 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 3503.02 msec. [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.534 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.554 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.575 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 0 rank id: 0 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.594 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.681 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 8118 to 8373 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.741 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.17 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.788 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.816 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 3503.47 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.845.844 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.018 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.039 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.068 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.101 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.121 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.846.138 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.847.882 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.847.905 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 8118 to 8373 [WARNING] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.847.964 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:188] Init] The port 8119 is already in use. So increase port to: 8119 [INFO] PS(619419,ffff237eefa0,python3.9):2025-04-03-09:35:40.848.198 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619419,ffff237eefa0,python3.9):2025-04-03-09:35:40.848.227 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.848.151 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:314b0634-2187-44e2-9413-e7cae5d10a37, the ip:127.0.0.1, the port:8119 [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.852.972 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.853.176 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 has been registered successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.853.206 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:40.853.755 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_5 to be registered, retry 19 times. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.142 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 0 has been started successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.206 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.266 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.287 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.334 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 0, global rank size: 8 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.386 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3008.51 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.409 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 0 is devserver-c233-2 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.539 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.624 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.649 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 0 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.667 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 0 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.694 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [0]. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.719 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.308 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.805 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.841 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.854.865 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:43.854.903 [device.cpp:365]619419 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.066 [api_impl.cc:249] 619419 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.086 [api_impl.cc:6795] 619419 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:43.855.100 [device.cpp:371]619419 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:43.855.189 [callback.cpp:37]619419 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:43.855.240 [callback.cpp:44]619419 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.855.270 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:43.855.297 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:43.855.345 [device.cpp:143]619419 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.445 [npu_driver.cc:124] 619419 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.482 [api_impl.cc:2999] 619419 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.526 [runtime.cc:3602] 619419 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.855.800 [hdc_core.c:1982][ascend][curpid:619419,619419][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4250; pid=619419) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.855.971 [runtime.cc:3609] 619419 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.856.031 [atrace_client_communication.c:103](tid:619419) send [hello] msg. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.856.055 [atrace_client_communication.c:63](tid:619419) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.856.205 [hdc_core.c:1982][ascend][curpid:619419,619419][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4251; pid=619419) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:43.856.227 [hdc_api.cpp:671][tid:619419] connect succ, peer_node: 0, peer_devid: 0 [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.857.595 [hdc_client.c:417][ascend][curpid:619419,619419][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4251) [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.857.632 [atrace_client_communication.c:65](tid:619419) send msg by adx[finish], ret=0. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.857.649 [atrace_client_communication.c:97](tid:619419) msgType=1, magic=49959, version=1. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.857.661 [atrace_client_communication.c:135](tid:619419) send [hello] msg success. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.857.744 [atrace_client_core.c:160](tid:619419) start to receive device[0] trace log. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.857.784 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.857.802 [runtime.cc:2855] 619419 startAicpuExecutor: TsdOpen devId=0, device mode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:43.857.817 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.857.874 [client_manager.cpp:160][GetPlatformInfo][tid:619419] [TsdClient] mode[1] chipType[5] [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.857.900 [hdc_core.c:1982][ascend][curpid:619419,619603][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4252; pid=619419) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:43.857.925 [hdc_api.cpp:671][tid:619603] connect succ, peer_node: 0, peer_devid: 0 [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.857.927 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619419] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.857.946 [client_manager.cpp:582][GetVisibleDevices][tid:619419] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.857.977 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.857.992 [client_manager.cpp:125][GetInstance][tid:619419] [ClientManager] GetInstance, deviceId[0], g_runningMode[0], begin saving instance [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.024 [client_manager.cpp:435][IsHostEnvironment][tid:619419] [TsdClient] logicDeviceId is [0], hostaicpunum[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.042 [client_manager.cpp:464][GetClientRunMode][tid:619419] runningMode:0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.055 [client_manager.cpp:127][GetInstance][tid:619419] [ClientManager] Current mode:1 [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.858.069 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619419] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.084 [client_manager.cpp:347][GetProfilingMode][tid:619419] Get AICPU_PROFILING_MODE[] [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.858.130 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619419] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.148 [process_mode_base.cpp:149][GetLogLevel][tid:619419] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.858.170 [atrace_client_communication.c:177](tid:619603) send [start] msg. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:43.858.222 [atrace_client_communication.c:189](tid:619603) send [start] msg success. [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.257 [process_mode_base.cpp:161][GetLogLevel][tid:619419] [TsdClient] Get Log Level[001] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.276 [process_mode_manager.cpp:71][OpenProcess][tid:619419] [ProcessModeManager] enter into open process deviceId[0] rankSize[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.295 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619419] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.371 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[.], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.465 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[scene.info], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.543 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.644 [client_manager.cpp:264][CheckPackageExists][tid:619419] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.662 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[ascend_install.info], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.737 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[version.info], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.808 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[..], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.858.892 [client_manager.cpp:261][CheckPackageExists][tid:619419] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.069 [client_manager.cpp:268][CheckPackageExists][tid:619419] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.090 [client_manager.cpp:283][CheckPackageExists][tid:619419] [TsdClient][deviceId=0] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.121 [client_manager.cpp:286][CheckPackageExists][tid:619419] [TsdClient][deviceId=0] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.859.195 [devdrv_hccl.c:103][ascend][curpid:619419,619419][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619419) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.213 [process_mode_manager.cpp:396][InitTsdClient][tid:619419] [TsdClient][deviceId=0] get process sign success, procPid[619419] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.228 [process_mode_manager.cpp:403][InitTsdClient][tid:619419] [TsdClient] deviceId[0] begin to init hdc client [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.270 [hdc_common.cpp:67][InitMsgSize][tid:619419] HdcCommon::InitMsgSize Start [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.287 [hdc_common.cpp:80][InitMsgSize][tid:619419] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.304 [hdc_client.cpp:184][CreateHdcSession][tid:619419] HdcClient::CreateHdcSession Start [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.393 [hdc_client.cpp:208][CreateHdcSession][tid:619419] [HdcClient] deviceId: 0 connect session [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.859.428 [hdc_core.c:1982][ascend][curpid:619419,619419][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4253; pid=619419) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.446 [hdc_client.cpp:219][CreateHdcSession][tid:619419] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.460 [hdc_client.cpp:230][CreateHdcSession][tid:619419] [HdcClient] deviceId: 0 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.480 [version_verify.cpp:36][SetVersionInfo][tid:619419] VersionVerify: send client version to server [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.542 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.574 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.591 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.859.645 [internal_api.cpp:101][GetScheduleEnv][tid:619419] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.859.981 [version_verify.cpp:68][PeerVersionCheck][tid:619419] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.860.005 [version_verify.cpp:89][ParseVersionInfo][tid:619419] VersionVerify: pass client version info success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.860.021 [hdc_client.cpp:275][CheckHdcConnection][tid:619419] Service[2] create hdc success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.860.041 [version_verify.cpp:122][SpecialFeatureCheck][tid:619419] VersionVerify: new type[35], supported [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.860.060 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619419] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.860.094 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619419] [TsdClient][deviceId=0] [sessionId=1] wait package info respond [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.860.113 [hdc_client.cpp:154][TsdRecvData][tid:619419] recv timeout is:150000 ms [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.143 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619419] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.174 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619419] [PackageInfoMsgProc] [sessionID=1] deviceId[0], checkCode[56684994] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.195 [hdc_client.cpp:172][TsdRecvData][tid:619419] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.210 [version_verify.cpp:122][SpecialFeatureCheck][tid:619419] VersionVerify: new type[64], supported [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.227 [hdc_client.cpp:411][Destroy][tid:619419] enter HdcClient::Destroy() function [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.861.244 [hdc_client.c:417][ascend][curpid:619419,619419][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4253) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.273 [hdc_client.cpp:394][DestroyClient][tid:619419] begin drvHdcClientDestroy [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.292 [hdc_client.cpp:399][DestroyClient][tid:619419] end drvHdcClientDestroy [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.306 [hdc_client.cpp:377][ClearClientPtr][tid:619419] begin HdcClient::ClearClientPtr [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.321 [hdc_client.cpp:385][ClearClientPtr][tid:619419] end HdcClient::ClearClientPtr [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.332 [hdc_client.cpp:418][Destroy][tid:619419] end HdcClient::Destroy() function [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.356 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619419] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.378 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619419] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.861.399 [devdrv_hccl.c:103][ascend][curpid:619419,619419][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619419) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.415 [process_mode_manager.cpp:396][InitTsdClient][tid:619419] [TsdClient][deviceId=0] get process sign success, procPid[619419] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.428 [process_mode_manager.cpp:403][InitTsdClient][tid:619419] [TsdClient] deviceId[0] begin to init hdc client [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.444 [hdc_common.cpp:67][InitMsgSize][tid:619419] HdcCommon::InitMsgSize Start [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.458 [hdc_common.cpp:80][InitMsgSize][tid:619419] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.472 [hdc_client.cpp:184][CreateHdcSession][tid:619419] HdcClient::CreateHdcSession Start [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.550 [hdc_client.cpp:208][CreateHdcSession][tid:619419] [HdcClient] deviceId: 0 connect session [INFO] DRV(619419,python3.9):2025-04-03-09:35:43.861.583 [hdc_core.c:1982][ascend][curpid:619419,619419][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4254; pid=619419) [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.601 [hdc_client.cpp:219][CreateHdcSession][tid:619419] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.616 [hdc_client.cpp:230][CreateHdcSession][tid:619419] [HdcClient] deviceId: 0 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.633 [version_verify.cpp:36][SetVersionInfo][tid:619419] VersionVerify: send client version to server [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.649 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.665 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.695 [version_verify.cpp:52][SetVersionInfo][tid:619419] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.916 [version_verify.cpp:68][PeerVersionCheck][tid:619419] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.936 [version_verify.cpp:89][ParseVersionInfo][tid:619419] VersionVerify: pass client version info success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.955 [hdc_client.cpp:275][CheckHdcConnection][tid:619419] Service[2] create hdc success [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.974 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619419] [TsdClient] tsd get process sign successfully, procpid[619419] signSize[48] [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.861.994 [version_verify.cpp:114][SpecialFeatureCheck][tid:619419] VersionVerify: previous type[6], supported [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.862.023 [process_mode_manager.cpp:88][OpenProcess][tid:619419] [ProcessModeManager] deviceId[0] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619419,python3.9):2025-04-03-09:35:43.862.042 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619419] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:43.862.060 [hdc_client.cpp:154][TsdRecvData][tid:619419] recv timeout is:150000 ms [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.059.541 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619419] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.059.565 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619419] [TsdClient] DeviceMsgProc recvMsg realDeviceId[0] msgType[8] localDevId[0] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.059.582 [hdc_client.cpp:172][TsdRecvData][tid:619419] [deviceId=0] the Tsd recv data already [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.059.610 [devmm_svm.c:132][ascend][curpid:619419,619419][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=0) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.061.803 [devmm_virt_interface.c:450][ascend][curpid:619419,619419][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.062.106 [drv_log_user.c:623][ascend][curpid:619419,619419][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=0; devid=0; vfid=0; hostpid=619419; devpid=319) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.062.132 [devmm_svm.c:122][ascend][curpid:619419,619419][drv][devmm][devmm_set_device_info]Device info. (devid=0; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.062.416 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619419] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.062.440 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619419] enter into OpenInHost deviceid[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.062.468 [client_manager.cpp:435][IsHostEnvironment][tid:619419] [TsdClient] logicDeviceId is [0], hostaicpunum[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.062.485 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619419] host cpu not support [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.062.499 [process_mode_manager.cpp:121][OpenProcess][tid:619419] [TsdClient][deviceId=0] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.062.516 [runtime.cc:2879] 619419 startAicpuExecutor: userDeviceId=0, devId=0, tsdOpen ret=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.062.544 [runtime.cc:2891] 619419 startAicpuExecutor: TsdOpen success, devId=0, ret=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.062.567 [runtime.cc:3627] 619419 DeviceRetain: new dev ok id=0, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.062.584 [raw_device.cc:936] 619419 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.062.602 [npu_driver.cc:2968] 619419 DeviceOpen: Open device start, device_id=0, tsId=0. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.062.703 [trs_sqcq.c:458][ascend][curpid:619419,619419][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=0) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.063.181 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=6; num=2044) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.063.669 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=7; num=2044) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.063.919 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=10; num=1024) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.189 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=11; num=1024) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.668 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=12; num=2048) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.699 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=8; num=16) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.721 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=9; num=16) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.738 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=13; num=4) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.064.758 [trs_sqcq.c:394][ascend][curpid:619419,619419][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=0; tsId=0; id_type=14; num=10) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.066.795 [trs_dev_drv.c:302][ascend][curpid:619419,619419][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=0; fd=37; mmapFd=38) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.066.819 [devmm_svm.c:132][ascend][curpid:619419,619419][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=0) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.066.837 [devmm_svm.c:122][ascend][curpid:619419,619419][drv][devmm][devmm_set_device_info]Device info. (devid=0; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.066.901 [npu_driver.cc:3011] 619419 DeviceOpen: Open device success, device_id=0, tsId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.066.922 [raw_device.cc:292] 619419 Init: chipType:0x5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.066.971 [raw_device.cc:304] 619419 Init: isAddrFlat:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.066.991 [raw_device.cc:320] 619419 Init: GetDevInfo, device_id=0, hardwareVersion=0x503 [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.067.008 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.067.026 [npu_driver.cc:1578] 619419 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.068.509 [raw_device.cc:352] 619419 Init: Get phy info, phy chip id 0, phy die id 0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.068.556 [runtime.cc:2771] 619419 GetTsdQos: [TSD] start get qos, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.068.571 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.587 [tsd_client.cpp:189][TsdCapabilityGet][tid:619419] TsdCapabilityGet Begin. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.602 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.615 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.628 [process_mode_manager.cpp:1042][CapabilityGet][tid:619419] [ProcessModeManager] enter into CapabilityGet process deviceId[0] type[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.644 [process_mode_manager.cpp:386][InitTsdClient][tid:619419] [TsdClient] tsd client has been inited before [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.663 [version_verify.cpp:114][SpecialFeatureCheck][tid:619419] VersionVerify: previous type[38], supported [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.699 [process_mode_manager.cpp:1065][CapabilityGet][tid:619419] [TsdClient][deviceId=0] [sessionId=1] [type=0]send capability success [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.068.720 [hdc_client.cpp:154][TsdRecvData][tid:619419] recv timeout is:150000 ms [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.069.820 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619419] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.069.841 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619419] [TsdClient] PidQosMsgProc recvMsg realDeviceId[0] msgType[39] localDevId[0] rspCode[0],pidqos[1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.069.857 [hdc_client.cpp:172][TsdRecvData][tid:619419] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.069.869 [process_mode_manager.cpp:1070][CapabilityGet][tid:619419] [TsdClient][logicDeviceId_=0][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.888 [runtime.cc:2782] 619419 GetTsdQos: [TSD] get qos success, userDeviceId=0, devId=0, qos=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.906 [engine.cc:78] 619419 Engine: Constructor. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.923 [stars_engine.cc:51] 619419 StarsEngine: Constructor. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.939 [engine_factory.cc:26] 619419 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.956 [engine_factory.cc:38] 619419 CreateEngine: Engine init success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.971 [raw_device.cc:364] 619419 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.069.983 [spm_pool.cc:42] 619419 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.070.113 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.070.133 [spm_pool.cc:55] 619419 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.070.149 [raw_device.cc:380] 619419 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.070.168 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.070.188 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.070.219 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.070.235 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.076.149 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.076.194 [task_allocator.cc:46] 619419 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.076.229 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.076.246 [h2d_copy_mgr.cc:45] 619419 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.076.260 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.076.274 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.170 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.193 [task_allocator.cc:46] 619419 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.207 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.223 [h2d_copy_mgr.cc:45] 619419 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.077.237 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.254 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.956 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.979 [task_allocator.cc:46] 619419 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.077.995 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.012 [h2d_copy_mgr.cc:45] 619419 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.027 [uma_arg_loader.cc:86] 619419 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.044 [task_allocator.cc:46] 619419 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.070 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.085 [uma_arg_loader.cc:93] 619419 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.886 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.907 [uma_arg_loader.cc:102] 619419 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.918 [uma_arg_loader.cc:104] 619419 Init: ALLOC PCIE is support[1] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.078.933 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.078.960 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.079 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.099 [h2d_copy_mgr.cc:45] 619419 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.114 [raw_device.cc:393] 619419 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.140 [raw_device.cc:398] 619419 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.158 [task_allocator.cc:46] 619419 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.331 [buffer_allocator.cc:61] 619419 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.512 [raw_device.cc:109] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.902 [raw_device.cc:121] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.920 [raw_device.cc:425] 619419 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.079.974 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.057 [stream.cc:2246] 619419 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.094 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.112 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.648 [npu_driver.cc:6125] 619419 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.754 [npu_driver.cc:6137] 619419 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=496, streamId=0, logicCq=496, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.790 [npu_driver.cc:6189] 619419 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=0, logicCq=496. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.810 [npu_driver.cc:5916] 619419 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0779000, addr=dfffe0779000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.828 [stream.cc:653] 619419 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.915 [stream.cc:276] 619419 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.935 [stream.cc:688] 619419 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=0, streamResId=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.957 [stream_state_callback_manager.cc:45] 619419 Notify: not reg stream state callback func! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.080.973 [raw_device.cc:593] 619419 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.006 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.033 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=16 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.050 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.157 [tsch_stream.cc:45] 619419 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=0 streamResId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.194 [raw_device.cc:1219] 619419 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.316 [osal.cc:309] 619419 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.444 [osal.cc:182] 619419 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.469 [osal.cc:309] 619419 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.524 [stars_engine.cc:1785] 619608 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.532 [osal.cc:182] 619419 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.549 [stars_engine.cc:201] 619609 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.572 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.081.591 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.081.612 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.081.625 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.649 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.702 [npu_driver.cc:6306] 619608 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.725 [stars_engine.cc:1855] 619608 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.743 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.915 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.942 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.081.985 [task_info.cc:1596] 619419 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.006 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.024 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.059 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.076 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.097 [stars_engine.cc:1122] 619419 SyncTask: Task Wait: device_id=0, stream_id=0, task_id=0, logicCqId=496. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.137 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.156 [stars_engine.cc:1071] 619419 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.205 [stars_engine.cc:1017] 619419 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.227 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.253 [stars_engine.cc:1027] 619419 ProcReport: taskType=97. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.280 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.296 [stars_engine.cc:1570] 619419 ProcLogicCqReport: CQE warning, device_id=0, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.332 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.349 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.366 [task_info.cc:1614] 619419 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.379 [raw_device.cc:1305] 619419 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.414 [stars_engine.cc:1351] 619419 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.438 [npu_driver.cc:5618] 619419 GetSqEnable: deviceId=0, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.455 [stars_engine.cc:1365] 619419 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.467 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.484 [npu_driver.cc:5595] 619419 GetSqTail: deviceId=0, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.500 [npu_driver.cc:5406] 619419 SetSqHead: deviceId=0, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.522 [npu_driver.cc:5357] 619419 EnableSq: deviceId=0, tsId=0, sqId=0. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.541 [stars_engine.cc:1430] 619419 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.560 [npu_driver.cc:5618] 619419 GetSqEnable: deviceId=0, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.574 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.588 [raw_device.cc:1331] 619419 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.082.601 [raw_device.cc:610] 619419 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.106 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.153 [ringbuffer_maintain_task.cc:98] 619419 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x2380 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.173 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.191 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.212 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.238 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.258 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.275 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.306 [stars_engine.cc:622] 619419 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=496, retCode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.324 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.338 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.353 [stream.cc:2886] 619419 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.370 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.387 [stream.cc:2886] 619419 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.401 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.417 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.433 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.447 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.467 [stars_engine.cc:437] 619419 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.482 [stream.cc:2886] 619419 StarsWaitForTask: No.3 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.522 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.543 [task_info.cc:111] 619419 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.561 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.577 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.599 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.617 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.634 [stars_engine.cc:1122] 619419 SyncTask: Task Wait: device_id=0, stream_id=0, task_id=2, logicCqId=496. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.770 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.812 [stars_engine.cc:1071] 619419 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.833 [stars_engine.cc:1017] 619419 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.874 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.893 [stars_engine.cc:1027] 619419 ProcReport: taskType=29. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.908 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.926 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.944 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.980 [npu_driver.cc:5618] 619419 GetSqEnable: deviceId=0, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.083.998 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.084.018 [raw_device.cc:1232] 619419 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.084.052 [atrace_api.c:40](tid:619419) AtraceCreate start. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.197 [trace_rb_log.c:45](tid:619419) init ring buffer time finished, realTime 1743644144084128662, monotonicTime 4211329407046, freq 99988 kHz, minutesWest 480 minutes [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.222 [trace_rb_log.c:137](tid:619419) [RUNTIME_ATRACE_DEV0_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.242 [tracer_mgr_operate.c:66](tid:619419) create object RUNTIME_ATRACE_DEV0_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.258 [atrace_api.c:44](tid:619419) AtraceCreate end. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.320 [trace_event.c:70](tid:619419) limitedNum 1, processType 0 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.085.337 [trace_event.c:372](tid:619419) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.362 [runtime.cc:3159] 619419 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619419,python3.9):2025-04-03-09:35:44.085.381 [client_manager.cpp:196][SetProfilingCallback][tid:619419] [TsdClient] set profiling callback success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.426 [runtime.cc:3364] 619419 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.470 [context.cc:109] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.537 [context.cc:121] 619419 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.554 [context.cc:276] 619419 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.637 [raw_device.cc:803] 619419 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.655 [npu_driver_dcache_lock.cpp:39] 619419 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.803 [raw_device.cc:528] 619419 AllocStackPhyBase: AllocStackPhyBase device_id=0, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.857 [raw_device.cc:232] 619419 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.887 [runtime.cc:3393] 619419 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.905 [runtime.cc:3560] 619419 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.943 [api_impl.cc:2987] 619419 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.961 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.085.977 [api_impl.cc:2992] 619419 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.086.005 [awatchdog_core.c:175](tid:619419) create watchdog for id : 65543, tid : 619419, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.086.024 [api_impl.cc:3025] 619419 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.086.069 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.086.089 [acl.cpp:121]619419 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.036 [api_impl.cc:3476] 619419 GetDeviceInfo: get device info, deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.077 [api_impl.cc:3476] 619419 GetDeviceInfo: get device info, deviceId=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.089.134 [device.cpp:133]619419 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 0, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.089.157 [device.cpp:154]619419 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.089.181 [context.cpp:87]619419 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.089.213 [context.cpp:98]619419 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.089.243 [stream.cpp:48]619419 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.289 [api_impl.cc:1120] 619419 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.353 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.388 [stream.cc:2246] 619419 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.434 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.451 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.902 [npu_driver.cc:6125] 619419 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.954 [npu_driver.cc:6137] 619419 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=497, streamId=2, logicCq=497, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.978 [npu_driver.cc:6189] 619419 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=2, logicCq=497. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.089.996 [npu_driver.cc:5916] 619419 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d8625000, addr=dfffd8625000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.090.011 [stream.cc:653] 619419 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.090.073 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.090.093 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.090.247 [stream.cc:291] 619419 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.092.964 [stream.cc:688] 619419 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=0, streamResId=65535 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.092.993 [stream_state_callback_manager.cc:45] 619419 Notify: not reg stream state callback func! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.011 [api_impl.cc:1162] 619419 StreamCreate: Succ, flags=1536, stream_id=2, context=aaaac1adc2f0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.093.035 [stream.cpp:189]619419 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.063 [stream.cc:3161] 619419 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.093.078 [stream.cpp:195]619419 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.093.124 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaac2b2c890. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.093.159 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.093.177 [stream.cpp:48]619419 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.197 [api_impl.cc:1120] 619419 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.235 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.270 [stream.cc:2246] 619419 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.312 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.329 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.570 [npu_driver.cc:6125] 619419 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.627 [npu_driver.cc:6137] 619419 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=498, streamId=3, logicCq=498, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.652 [npu_driver.cc:6189] 619419 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=3, logicCq=498. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.670 [npu_driver.cc:5916] 619419 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d8604000, addr=dfffd8604000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.686 [stream.cc:653] 619419 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.093.733 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.093.752 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.061 [stream.cc:291] 619419 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.698 [stream.cc:688] 619419 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=0, streamResId=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.720 [stream_state_callback_manager.cc:45] 619419 Notify: not reg stream state callback func! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.752 [api_impl.cc:1162] 619419 StreamCreate: Succ, flags=1536, stream_id=3, context=aaaac1adc2f0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.094.767 [stream.cpp:189]619419 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.786 [stream.cc:3161] 619419 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.094.801 [stream.cpp:195]619419 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.094.835 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaac2c5d1a0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.094.936 [callback.cpp:25]619419 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471259520928. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.094.963 [npu_driver.cc:462] 619419 EventIdAlloc: event id alloc begin, device_id=0, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.094.976 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.012 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.028 [npu_driver.cc:483] 619419 EventIdAlloc: Event id alloc success, device_id=0, ts_id=0, event_id=0, eventFlag=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.031 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.043 [event.cc:347] 619419 GenEventId: Event id alloc success, device_id=0, tsId=0, eventFlag=1, event_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.062 [subscribe.cc:54] 619419 AssignGroupId: give groupId=64 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.079 [npu_driver.cc:3775] 619419 SqCqAllocate: deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.100 [npu_driver.cc:3785] 619419 SqCqAllocate: alloc success, deviceId=0 sqId=135, cqId=135. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.112 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.120 [subscribe.cc:131] 619419 Insert: callback subscribe: groupId=64, threadId=281471259520928, devId=0, tsId=0, streamId=2, sqId=135, cqId=135 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.136 [callback.cpp:31]619419 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471259520928. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.150 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.167 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaac2b2c890. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.171 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.241 [callback.cpp:25]619419 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471251066784. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.265 [npu_driver.cc:462] 619419 EventIdAlloc: event id alloc begin, device_id=0, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.266 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.285 [npu_driver.cc:483] 619419 EventIdAlloc: Event id alloc success, device_id=0, ts_id=0, event_id=1, eventFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.317 [event.cc:347] 619419 GenEventId: Event id alloc success, device_id=0, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.317 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.331 [subscribe.cc:54] 619419 AssignGroupId: give groupId=65 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.338 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.345 [npu_driver.cc:3775] 619419 SqCqAllocate: deviceId=0, tsId=0, groupId=65. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.353 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.362 [npu_driver.cc:3785] 619419 SqCqAllocate: alloc success, deviceId=0 sqId=136, cqId=136. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.367 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.378 [subscribe.cc:131] 619419 Insert: callback subscribe: groupId=65, threadId=281471251066784, devId=0, tsId=0, streamId=3, sqId=136, cqId=136 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.095.396 [callback.cpp:31]619419 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471251066784. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.401 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.425 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaac2c5d1a0. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.448 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.531 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.095.857 [npu_driver.cc:2616] 619419 MemGetInfoEx: memory info type=1, type=2, free=8860618752, hug_free=56245616640 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.902 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.095.995 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.096.041 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.096.070 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.096.093 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62090M, Reserved MOC size for Other Components(HCCL/rts/etc.):3892M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58198M. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.096.131 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58198M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.096.175 [event.cpp:260]619419 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.096.199 [api_impl.cc:5713] 619419 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.276 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.298 [timeout_set_task.cc:115] 619419 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.314 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.330 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.350 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.365 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.382 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.399 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.431 [stars_engine.cc:622] 619419 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=496, retCode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.448 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.461 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.479 [stream.cc:2886] 619419 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.493 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.507 [stream.cc:2886] 619419 StarsWaitForTask: No.2 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.523 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.537 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.553 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.565 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.583 [stars_engine.cc:437] 619419 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.600 [stream.cc:2886] 619419 StarsWaitForTask: No.3 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.097.612 [event.cpp:262]619419 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.097.630 [event.cpp:269]619419 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.666 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.684 [timeout_set_task.cc:115] 619419 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.699 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.712 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.731 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.747 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.764 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.776 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.791 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.803 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.816 [stream.cc:2886] 619419 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.832 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.845 [stream.cc:2886] 619419 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.859 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.871 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.886 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.897 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.914 [stars_engine.cc:437] 619419 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.097.931 [stream.cc:2886] 619419 StarsWaitForTask: No.3 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.097.945 [event.cpp:277]619419 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.097.976 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.014 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.033 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 0 has been set [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.050 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.076 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.131 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.162 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.098.213 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.339 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.098.364 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.405 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.431 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.098.451 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.469 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.630 [acl.cpp:288]619419 aclInit: start to execute aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.674 [acl.cpp:298]619419 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.098.791 [acl.cpp:242]619419 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.107.773 [error_manager.cc:673]619419 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.108.539 [acl.cpp:315]619419 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:44.108.624 [prof_atls_plugin.cpp:210] (tid:619419) Module[45] register callback of ctrl handle. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.108.675 [ge_executor.cc:273]619419 Initialize:Init GeExecutor begin. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.108.743 [op_lib_register.cc:117]619419 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.108.827 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.108.876 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.108.910 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.108.944 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.108.975 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.007 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.039 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.072 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.111 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.145 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.175 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.206 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.237 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.267 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.297 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.332 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.363 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.393 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.424 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.109.455 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.496 [op_lib_register.cc:165]619419 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.538 [plugin_manager.cc:616]619419 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.555 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.587 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.612 [plugin_manager.cc:624]619419 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.109.629 [plugin_manager.cc:206]619419 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.140 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.184 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.202 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.215 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.261 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.623 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.660 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.677 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.693 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.731 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.783 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.801 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.110.816 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.831 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.110.850 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.208 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.243 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.260 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.287 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.324 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.685 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.721 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.738 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.754 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.790 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.834 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.852 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.111.868 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.882 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.111.898 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.262 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.302 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.321 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.336 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.375 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.728 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.778 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.796 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.815 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.851 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.897 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.914 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.112.928 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.942 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.112.959 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.310 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.344 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.362 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.375 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.410 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.755 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.790 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.808 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.824 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.859 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.907 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.925 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.113.940 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.953 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.113.968 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.308 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.344 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.361 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.375 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.410 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.754 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.788 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.806 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.821 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.855 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.891 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.907 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.114.933 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.949 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.967 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.114.984 [plugin_manager.cc:218]619419 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.004 [plugin_manager.cc:422]619419 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.019 [plugin_manager.cc:155]619419 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.041 [plugin_manager.cc:158]619419 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.055 [plugin_manager.cc:426]619419 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.390 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.115.424 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.115.441 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.115.456 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.492 [plugin_manager.cc:439]619419 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.524 [plugin_manager.cc:175]619419 ReversePathString:Enter ReversePathString schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.548 [plugin_manager.cc:187]619419 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.873 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.905 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.115.928 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.115.971 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619419,python3.9):2025-04-03-09:35:44.131.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.131.643 [op_tiling_registry.cc:162]619419 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.131.760 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.131.857 [op_tiling_registry.cc:162]619419 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619419,python3.9):2025-04-03-09:35:44.131.961 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.060 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.173 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.284 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.306 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.328 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.426 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.518 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.608 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.718 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619419,python3.9):2025-04-03-09:35:44.132.815 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.134 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.224 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.326 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.424 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.523 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.627 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.733 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.133.914 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.014 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.122 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.227 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.246 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.344 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.448 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.467 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.560 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.659 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.758 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.134.951 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.123 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.218 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.336 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.444 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.857 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.135.971 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.081 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.283 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.395 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.507 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.612 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.712 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.809 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.136.912 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.018 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.114 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.134 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.235 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.338 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.437 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.458 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.553 [op_tiling_registry.cc:122]619419 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.653 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.756 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.860 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619419,python3.9):2025-04-03-09:35:44.137.977 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.170 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.258 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.352 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.428 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.505 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.664 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.692 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.787 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.841 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.867 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.138.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.065 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.143 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.221 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.298 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.382 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.410 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.481 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.515 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.543 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.616 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.714 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.795 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.869 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.139.944 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.020 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.097 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.189 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.269 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.344 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.453 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.534 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.695 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.772 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.848 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.140.925 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.144.914 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619419,python3.9):2025-04-03-09:35:44.144.949 [op_tiling_registry.cc:148]619419 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619419,python3.9):2025-04-03-09:35:44.146.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.146.423 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.146.537 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.146.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.146.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.055 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.323 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.435 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.729 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.845 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.147.957 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.148.081 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.403 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.540 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.578 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.738 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.848 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.150.989 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.103 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.218 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.250 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.360 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.473 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.624 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.739 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.870 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.151.986 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.111 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.246 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.364 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.485 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.596 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.714 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.745 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.152.955 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.225 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.441 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.759 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.153.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.028 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.058 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.177 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.207 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.319 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.425 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.454 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.579 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.690 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.801 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.154.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.117 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.149 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.370 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.566 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.675 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.155.906 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.021 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.227 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.384 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.693 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.820 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.156.933 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.049 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.155 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.267 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.525 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.706 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.765 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffefd345d70, registered count:1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.936 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.157.959 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffefd34e248, registered count:2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.195 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.341 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.576 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.692 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.803 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.158.920 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.037 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.149 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.178 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.292 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.399 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.518 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.627 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.740 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.855 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.159.964 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.252 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.289 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.347 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.493 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.609 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.724 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.160.946 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.078 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.222 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.336 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.452 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.642 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.756 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.161.979 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.099 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.210 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.322 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.474 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.162.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.005 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.118 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.235 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.348 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.463 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.575 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.689 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.163.883 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.164.000 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.164.121 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.166.627 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.166.676 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.166.707 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.166.840 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.166.958 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.186 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.297 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.408 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.521 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.652 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.773 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.801 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.921 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.167.950 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.071 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.100 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.222 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.331 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.444 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.555 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.168.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.169.061 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.169.207 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.169.325 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.169.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.169.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.170.211 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.171.003 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.171.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.171.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.172.367 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.172.494 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.172.632 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.172.784 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.172.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.173.179 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.175.338 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.176.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.176.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.176.503 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.177.464 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.177.620 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.177.778 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.231 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.573 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.685 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.809 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.178.928 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.041 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.152 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.264 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.391 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.552 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.681 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.798 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.179.985 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.107 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.238 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.356 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.684 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.800 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.180.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.134 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.252 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.375 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.546 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.665 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.831 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.181.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.065 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.185 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.215 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.327 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.365 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.478 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.594 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.623 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.853 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.182.971 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.154 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.271 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.469 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.620 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.799 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.183.982 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.193 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.380 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.500 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.627 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.741 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.851 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.184.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.074 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.186 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.430 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.547 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.658 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.767 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.799 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.185.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.186.150 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.187.384 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.189.813 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.190.020 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.190.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.190.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.190.709 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.190.862 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.191.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.191.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.191.958 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.108 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.281 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.428 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.746 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.192.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.268 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.577 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.610 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.640 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.672 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.697 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.723 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.749 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.777 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.804 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.831 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.857 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.882 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.912 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.939 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.965 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.193.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.018 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.047 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.097 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.136 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.188 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.214 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.243 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.271 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.321 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.348 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.400 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.427 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.481 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.534 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.614 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.647 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.673 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.701 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.728 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.844 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.870 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.896 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.921 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.194.975 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.001 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.084 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.250 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.406 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.583 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.613 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.640 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.666 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.835 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.195.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.076 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.206 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.331 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.444 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.615 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.737 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.196.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.151 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.181 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.209 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.333 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.361 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.387 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.505 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.664 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.837 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.197.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.111 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.423 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.752 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.872 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.912 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.198.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.111 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.290 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.444 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.597 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.199.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.427 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.617 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.756 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.876 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.200.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.124 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.419 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.578 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.698 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.201.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.202.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.202.200 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.202.658 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.120 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.243 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.366 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.398 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.525 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.644 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.762 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.203.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.012 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.273 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.403 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.684 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.807 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.204.998 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.126 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.258 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.374 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.570 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.770 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.205.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.009 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.263 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.389 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.625 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.206.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.207.075 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.207.336 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.207.539 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.207.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.207.901 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.132 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.367 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.602 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.630 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.657 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.680 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.716 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.752 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.208.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.209.573 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.210.001 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.210.422 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.210.621 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.210.837 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.211.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.211.391 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.211.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.214.324 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.216.170 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.216.333 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.216.459 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.216.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.216.881 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.049 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.242 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.622 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.693 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.719 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.217.948 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.120 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.298 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.467 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.642 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.846 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.218.880 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.051 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.401 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.587 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.757 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.219.948 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.097 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.313 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.347 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.464 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.498 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.624 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.664 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.797 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.826 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.220.951 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.078 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.342 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.457 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.570 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.774 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.221.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.222.372 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.222.772 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.222.981 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.514 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.812 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.847 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.958 [op_tiling_registry.cc:134]619419 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.223.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.224.141 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.224.198 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.224.232 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.224.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.225.399 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.225.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.015 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.330 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.363 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.509 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.658 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.808 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.226.843 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.227.003 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.227.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.227.313 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.227.487 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.227.644 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.228.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.228.425 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.228.670 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.228.867 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.229.923 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.230.394 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.230.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.127 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.285 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.582 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.734 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.768 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.231.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.232.282 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.232.410 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.232.574 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.232.739 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.232.869 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.387 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.541 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.662 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.233.952 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.361 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.531 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.561 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.733 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.234.923 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.236.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.242.128 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.242.182 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.244.988 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.245.261 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.245.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.245.890 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.091 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.294 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.807 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.246.957 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.247.045 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.247.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.247.429 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.247.581 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.049 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.246 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.415 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.452 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.699 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.248.882 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.079 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.282 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.579 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.700 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.249.730 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.250.514 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.250.733 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.250.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.013 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.340 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.454 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.483 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.509 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.654 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.876 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.251.969 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.222 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.431 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.464 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.492 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.689 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.252.722 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.520 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.616 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.644 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.715 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.795 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.818 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.255.891 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.044 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.064 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.191 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.218 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.239 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.262 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.440 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.256.461 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.257.640 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffefdc98a10, registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.257.690 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffefdc98c70, registered count:1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.258.114 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.258.594 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.260.308 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.260.477 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.260.564 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.260.978 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.261.443 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.292.363 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.292.498 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.292.680 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.292.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.292.901 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.293.042 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.293.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.293.746 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.293.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.294.083 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.294.234 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.294.470 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.294.638 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.295.068 [op_tiling_manager.cc:79]619419 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.152 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.371 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.422 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.487 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.542 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.584 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.629 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.295.675 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.296.700 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.296.743 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.296.766 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.296.798 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.296.822 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.159 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff216a69b0, registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.186 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.206 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.225 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.244 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.558 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff216a9da0, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.583 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.601 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.620 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.639 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.297.936 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.297.993 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.298.079 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.298.849 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff216602c0, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.298.884 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.298.915 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.298.935 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.298.953 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.288 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff21664978, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.313 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.332 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.347 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.364 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.666 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.299.718 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.299.794 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.299.845 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.299.888 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.299.935 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.299.970 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.024 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.069 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.300.105 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.144 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.202 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.249 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.289 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.331 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.300.364 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.402 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.442 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.300.487 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.526 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.567 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.605 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.644 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.684 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.300.716 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.754 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.794 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.300.827 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.875 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.916 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.956 [op_tiling_manager.cc:73]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.300.994 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.301.034 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.067 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.301.105 [op_tiling_manager.cc:58]619419 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.301.144 [op_tiling_manager.cc:67]619419 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.176 [op_tiling_manager.cc:75]619419 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.196 [op_tiling_manager.cc:109]619419 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [191659] micro seconds. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.340 [plugin_manager.cc:639]619419 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.362 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.395 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.301.476 [host_cpu_engine.cc:254]619419 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.306.869 [host_cpu_engine.cc:273]619419 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619419,python3.9):2025-04-03-09:35:44.306.938 [host_cpu_engine.cc:254]619419 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.313.215 [host_cpu_engine.cc:267]619419 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619419,python3.9):2025-04-03-09:35:44.314.031 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619419] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.066 [folding.cc:404][InitCpuConstantFoldingNew][tid:619419]:"Init cpu constant folding begin." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.103 [folding.cc:408][InitCpuConstantFoldingNew][tid:619419]:"Number of registered ops: 234" [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.121 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Acos]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.144 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Acosh]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.161 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.178 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.194 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.209 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.222 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.238 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Add]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.255 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Addcdiv]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.271 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Addcmul]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.284 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[AffineGrid]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.299 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ApplyAdamV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.311 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Asin]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.327 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Asinh]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.341 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Assert]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.357 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Atan]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.374 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Atan2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.388 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Atanh]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.401 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[AvgPoolV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.433 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Betainc]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.449 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Bucketize]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.463 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CTCLossV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.479 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.492 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CacheSwapTable]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.504 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.518 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CaseCondition]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cast]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.550 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Ceil]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.566 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CheckNumerics]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.582 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ClipByValue]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.599 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ClipByValueV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.611 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Combinations]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.626 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CompareAndBitpack]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.638 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.654 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ConcatV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.670 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Conj]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.686 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ConjugateTranspose]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.702 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.718 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cosh]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.734 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[CropAndResize]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.752 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cross]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.768 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cummax]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.780 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cummin]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.795 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Cumsum]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.807 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Div]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.823 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[DynamicStitch]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.840 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EditDistance]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.861 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Elu]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.875 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EluGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.892 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.906 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.921 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.938 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.951 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.964 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.982 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.314.997 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.011 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.026 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.039 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.053 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.067 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.081 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.095 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingLookup]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.110 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.123 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.138 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.151 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.167 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.179 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Equal]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.191 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Expand]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.204 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ExpandDims]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.219 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Exponential]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.231 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.247 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Fill]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.269 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[FillDiagonal]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.280 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Fills]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.295 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Floor]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.308 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[FloorDiv]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.323 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[FloorMod]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.340 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GEMM]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.357 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GatherElements]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.372 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GatherV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.388 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Geometric]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.401 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GetDynamicDims]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.416 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Greater]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.428 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GreaterEqual]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.443 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GridSampler2D]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.460 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.477 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GridSampler3D]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.494 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.508 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Histogram]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.524 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.538 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Identity]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.552 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[IndexToAddr]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.570 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InitPartitionMap]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.603 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.616 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.631 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InvGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[InvertPermutation]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[IsClose]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.675 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[IsInf]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.694 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[IsNan]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.706 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Less]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.722 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[LinSpace]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.734 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[LinearSumAssignment]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.749 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Log]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.764 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Log1p]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.777 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.792 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.808 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.824 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MaskedFill]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.840 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MaskedScatter]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.852 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MaskedSelect]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.868 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.882 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MatMul]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.897 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.915 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MatrixDiagV3]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.928 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MinAreaPolygons]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.943 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Mul]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.957 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.970 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.985 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.315.998 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NLLLoss]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.013 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NLLLossGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.031 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NMSWithMask]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.044 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Neg]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.056 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.070 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NonZero]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.086 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.103 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.124 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[NotEqual]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.139 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[OnesLike]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.155 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[PSAMask]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.192 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[PSAMaskGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.210 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Pad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.224 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[PadD]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.238 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[PadV3]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.253 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[PadV3Grad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.266 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Pinverse]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.281 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Poisson]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.294 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Pow]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.310 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.326 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RNNTLoss]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.341 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RandomStandardNormal]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.358 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RandomUniform]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.373 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RealDiv]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.387 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ReduceAll]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.400 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ReduceMean]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.415 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ReduceSum]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.432 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Reshape]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.449 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Resize]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.466 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ResizeBilinear]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.478 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.493 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.505 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ResizeGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.520 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ResizeTrilinear]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.532 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ReverseSequence]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.547 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RightShift]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.571 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.583 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[RotatedNMS]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.599 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Round]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.616 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Rsqrt]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.628 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[STFT]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.660 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ScatterElements]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.677 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ScatterNdAdd]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.691 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ScatterNdMax]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.707 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ScatterNdMin]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.723 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SearchSorted]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.737 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SelectV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.752 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Sigmoid]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.766 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SigmoidGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.784 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Sign]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.801 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SilentCheck]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.818 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SilentCheckV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.831 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SilentCheckV3]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.846 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SliceWrite]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.858 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SoftmaxV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.874 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Sort]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.886 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.901 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SparseSegmentMean]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.916 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.932 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SparseSegmentSum]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.949 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SparseToDense]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.961 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SpatialTransformer]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.976 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SplitD]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.316.988 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SplitV]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.008 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Sqrt]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.020 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SqrtGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.033 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Square]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.047 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[SquaredDifference]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.061 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[StatelessBernoulli]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.076 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.088 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[StatelessRandperm]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.104 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[StridedSlice]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.116 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[StridedSliceV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.133 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Sub]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.146 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TableToResource]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.162 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Tan]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.174 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TanhGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.189 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TensorEqual]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.201 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.217 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ThreeNN]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.229 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Tile]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.244 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TileWithAxis]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.258 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TopK]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.273 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TopKPQDistance]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.285 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.300 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TopKV2D]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.314 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Trace]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.329 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TransData]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.347 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[TransDataRNN]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.361 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Tril]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.374 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Triu]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.389 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.401 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.421 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UninitPartitionMap]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.433 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UniqueConsecutive]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.449 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.465 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.483 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.496 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.511 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.527 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.544 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ViewCopy]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.560 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Voxelization]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.574 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[WarpAffine]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.589 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[ZerosLike]." [INFO] AICPU(619419,python3.9):2025-04-03-09:35:44.317.601 [folding.cc:413][InitCpuConstantFoldingNew][tid:619419]:"Register op[Zeta]." [INFO] OP(619419,python3.9):2025-04-03-09:35:44.317.621 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619419] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619419,python3.9):2025-04-03-09:35:44.317.637 [host_cpu_engine.cc:273]619419 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619419,python3.9):2025-04-03-09:35:44.317.778 [plugin_manager.cc:872]619419 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.321.239 [plugin_manager.cc:883]619419 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.317 [ops_kernel_executor_manager.cc:37]619419 Initialize:No library to load [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.339 [plugin_manager.cc:540]619419 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.355 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.390 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.412 [plugin_manager.cc:547]619419 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.321.432 [plugin_manager.cc:206]619419 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.322.319 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.366 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.384 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.401 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.322.450 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.322.801 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.837 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.854 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.871 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.322.906 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.948 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.966 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.322.981 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.322.998 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.031 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.046 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.073 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.323.089 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.323.431 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.467 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.483 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.502 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.323.543 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.323.894 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.929 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.946 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.323.959 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.323.995 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.032 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.049 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.062 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.324.077 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.106 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.122 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.154 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.324.189 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.324.537 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.573 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.590 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.324.607 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.324.643 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.324.992 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.034 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.051 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.063 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.325.100 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.137 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.152 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.170 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.325.187 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.219 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.234 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.263 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.325.276 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.325.616 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.653 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.670 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.325.682 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.325.718 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.063 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.097 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.114 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.132 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.170 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.206 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.223 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.236 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.252 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.295 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.322 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.335 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.349 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.700 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.735 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.751 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.326.764 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.326.800 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.148 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.182 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.198 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.211 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.248 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.293 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.309 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.321 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.336 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.374 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.404 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.420 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.436 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.447 [plugin_manager.cc:218]619419 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.469 [plugin_manager.cc:422]619419 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.486 [plugin_manager.cc:155]619419 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.509 [plugin_manager.cc:158]619419 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.521 [plugin_manager.cc:426]619419 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.853 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.889 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.905 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.327.917 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.956 [plugin_manager.cc:439]619419 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619419,python3.9):2025-04-03-09:35:44.327.995 [ge_executor.cc:259]619419 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.374 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.414 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.437 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.548 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.567 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.645 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.662 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.328.694 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.744 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.760 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.328.789 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.835 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.852 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.935 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.328.951 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.016 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.031 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.065 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.111 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.127 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.168 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.220 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.236 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.321 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.337 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.410 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.427 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.457 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.501 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.518 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.547 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.603 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.620 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.685 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.701 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.764 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.780 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.809 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.855 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.872 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.329.902 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.944 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.329.961 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.033 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.049 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.128 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.147 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.330.180 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.230 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.247 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.330.277 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.321 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.337 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.330.370 [opsproto_manager.cc:123]619419 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.417 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.487 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.503 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.520 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.537 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.330.552 [opsproto_manager.cc:147]619419 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.884 [operator_factory_impl.cc:215]619419 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.911 [operator_factory_impl.cc:269]619419 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619419,python3.9):2025-04-03-09:35:44.330.945 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.337.128 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619419,python3.9):2025-04-03-09:35:44.337.943 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.029 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.075 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.125 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.159 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.189 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.219 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.248 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.338.277 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.357.902 [operator_factory_impl.cc:368]619419 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619419,python3.9):2025-04-03-09:35:44.358.141 [operator_factory_impl.cc:284]619419 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.333 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.357 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.374 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.387 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.405 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619419,python3.9):2025-04-03-09:35:44.358.429 [operator_factory_impl.cc:254]619419 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619419,python3.9):2025-04-03-09:35:44.358.457 [operator_factory_impl.cc:340]619419 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619419,python3.9):2025-04-03-09:35:44.358.484 [operator_factory_impl.cc:307]619419 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.515 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.538 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.559 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.358.675 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.535 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.580 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.607 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.627 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.649 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.667 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.690 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.729 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.749 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.763 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.783 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.810 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.827 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.858 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.889 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.913 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.359.984 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.360.027 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.360.046 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.360.063 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.360.097 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.360.121 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.355 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.388 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.410 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.430 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.447 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.481 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.505 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.545 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.562 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.577 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.675 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.693 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.713 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.729 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.746 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.361.761 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.085 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.106 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.123 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.142 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.157 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.170 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.189 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.219 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.238 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.257 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.275 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.296 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.314 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.547 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.579 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.636 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.653 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.671 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.688 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.362.703 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.634 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.664 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.683 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.705 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.726 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.743 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.760 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.885 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.363.971 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.364 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.389 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.406 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.421 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.438 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.453 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.364.938 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.256 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.281 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.301 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.336 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.502 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.655 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.952 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.365.978 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.020 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.037 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.624 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.672 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.693 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.717 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.737 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.757 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.781 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.803 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.860 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.366.901 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.367.311 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.367.506 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.367.791 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.367.824 [operator_factory_impl.cc:222]619419 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.469 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.513 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.541 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.566 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.604 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.632 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.659 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.685 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.709 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.736 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.764 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.791 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.842 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.867 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.892 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.966 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.368.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.012 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.036 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.079 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.131 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.156 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.181 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.210 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.275 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.320 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.347 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.375 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.400 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.424 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.449 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.473 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.498 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.523 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.547 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.573 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.601 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.653 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.678 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.702 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.725 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.750 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.775 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.796 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.817 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.844 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.874 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.927 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.954 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.369.980 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.004 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.083 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.139 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.188 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.213 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.262 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.287 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.314 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.339 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.364 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.388 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.413 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.438 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.464 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.488 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.517 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.542 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.564 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.589 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.635 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.666 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.716 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.740 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.765 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.789 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.839 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.920 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.943 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.967 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.370.992 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.017 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.043 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.067 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.091 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.115 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.145 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.170 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.194 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.244 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.269 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.294 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.369 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.395 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.421 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.445 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.472 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.496 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.520 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.544 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.568 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.590 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.612 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.646 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.673 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.694 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.715 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.737 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.840 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.865 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.890 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.916 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.937 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.371.982 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.048 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.068 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.089 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.112 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.287 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.370 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.395 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.420 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.446 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.478 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.503 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.528 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.552 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.577 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.604 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.631 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.681 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.708 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.732 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.756 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.781 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.829 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.854 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.953 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.372.980 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.005 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.030 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.138 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.187 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.212 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.239 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.268 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.324 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.349 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.374 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.398 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.422 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.446 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.473 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.498 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.522 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.546 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.569 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.595 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.619 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.646 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.694 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.770 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.796 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.843 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.869 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.890 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.914 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.938 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.962 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.373.986 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.011 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.040 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.063 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.087 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.166 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.284 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.395 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.506 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.613 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.746 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.773 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.837 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.374.965 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.069 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.175 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.294 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.399 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.525 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.642 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.694 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.721 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.822 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.936 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.962 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.375.986 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.008 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.030 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.056 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.311 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.436 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.558 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.706 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.813 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.838 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.862 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.884 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.376.985 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.097 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.337 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.445 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.550 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.577 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.686 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.796 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.377.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.013 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.037 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.148 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.261 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.479 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.602 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.735 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.762 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.842 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.953 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.378.978 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.003 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.108 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.215 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.240 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.262 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.284 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.305 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.327 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.444 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.550 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.670 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.695 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.914 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.942 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.379.965 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.211 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.328 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.436 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.548 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.658 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.770 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.876 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.380.984 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.088 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.205 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.232 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.368 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.475 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.500 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.607 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.722 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.827 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.381.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.045 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.073 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.187 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.295 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.402 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.510 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.642 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.758 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.865 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.382.970 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.078 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.190 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.240 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.358 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.384 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.407 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.429 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.451 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.486 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.507 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.549 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.607 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.630 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.674 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.696 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.716 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.821 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.383.927 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.040 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.144 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.262 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.393 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.507 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.536 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.584 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.734 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.760 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.787 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.939 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.384.960 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.064 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.287 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.390 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.497 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.599 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.703 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.847 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.385.953 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.285 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.399 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.428 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.456 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.482 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.595 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.700 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.386.919 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.473 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.583 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.698 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.913 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.387.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.049 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.177 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.234 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.262 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.285 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.314 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.337 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.359 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.469 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.510 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.624 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.729 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.834 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.388.948 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.179 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.290 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.419 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.529 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.637 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.744 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.858 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.389.964 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.078 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.159 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.184 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.212 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.266 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.292 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.329 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.352 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.377 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.403 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.430 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.482 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.532 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.665 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.773 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.806 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.390.920 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.028 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.134 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.220 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.328 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.555 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.749 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.860 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.391.973 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.075 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.207 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.324 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.441 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.555 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.580 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.682 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.793 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.392.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.016 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.126 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.152 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.254 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.474 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.578 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.682 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.797 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.393.921 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.030 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.479 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.584 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.609 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.714 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.829 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.394.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.054 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.162 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.276 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.390 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.496 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.607 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.715 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.826 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.931 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.395.956 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.068 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.190 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.304 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.410 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.523 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.641 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.396.938 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.044 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.152 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.273 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.387 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.501 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.715 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.827 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.852 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.397.965 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.178 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.274 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.295 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.316 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.336 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.357 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.457 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.495 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.519 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.620 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.732 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.845 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.398.951 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.060 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.085 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.108 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.315 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.339 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.448 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.553 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.660 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.768 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.792 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.817 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.841 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.946 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.974 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.399.996 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.115 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.177 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.292 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.316 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.338 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.360 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.571 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.677 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.702 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.725 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.400.964 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.075 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.181 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.206 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.228 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.329 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.376 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.495 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.603 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.707 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.401.927 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.139 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.253 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.469 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.591 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.696 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.807 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.402.919 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.161 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.185 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.291 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.315 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.341 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.449 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.681 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.790 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.403.901 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.010 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.124 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.245 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.355 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.468 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.494 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.517 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.539 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.778 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.404.897 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.117 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.225 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.332 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.446 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.549 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.659 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.764 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.405.982 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.103 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.430 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.541 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.762 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.406.973 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.087 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.303 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.409 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.512 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.536 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.758 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.407.873 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.101 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.231 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.343 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.456 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.579 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.685 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.794 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.408.906 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.019 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.123 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.234 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.342 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.451 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.559 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.678 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.409.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.113 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.227 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.661 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.410.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.149 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.186 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.231 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.266 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.297 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.331 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.389 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.417 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.462 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.484 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.507 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.540 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.566 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.673 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.790 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.411.896 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.001 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.111 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.246 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.449 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.480 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.599 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.706 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.818 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.412.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.157 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.183 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.206 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.227 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.392 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.426 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.456 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.490 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.522 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.555 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.587 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.647 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.678 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.719 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.747 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.771 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.808 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.852 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.881 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.911 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.413.938 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.015 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.132 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.159 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.273 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.379 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.485 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.593 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.842 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.414.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.060 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.269 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.307 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.411 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.520 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.626 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.730 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.415.891 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.124 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.249 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.371 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.480 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.659 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.754 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.416.976 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.081 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.295 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.325 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.431 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.553 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.659 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.772 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.417.987 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.134 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.243 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.351 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.569 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.723 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.755 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.783 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.838 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.418.998 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.104 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.189 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.243 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.349 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.454 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.681 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.789 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.905 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.931 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.419.954 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.096 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.122 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.219 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.341 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.557 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.664 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.771 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.420.998 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.107 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.191 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.219 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.328 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.546 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.659 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.780 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.884 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.421.995 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.079 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.107 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.336 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.958 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.422.985 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.011 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.039 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.064 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.088 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.113 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.159 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.184 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.210 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.236 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.261 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.372 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.594 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.746 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.772 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.423.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.096 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.214 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.346 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.375 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.398 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.420 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.443 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.549 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.652 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.767 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.883 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.912 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.424.939 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.081 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.196 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.305 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.425 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.528 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.636 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.663 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.777 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.802 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.915 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.943 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.970 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.425.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.023 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.074 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.105 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.130 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.155 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.207 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.233 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.286 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.392 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.417 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.451 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.574 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.601 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.626 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.653 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.677 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.704 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.732 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.757 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.783 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.426.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.427.912 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.018 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.109 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.292 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.372 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.491 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.598 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.708 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.789 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.870 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.962 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.428.987 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.347 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.426 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.511 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.590 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.429.959 [opsproto_manager.cc:162]619419 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.011 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.055 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.492 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.548 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.571 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.649 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.668 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.736 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.753 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.766 [opp_so_manager.cc:56]619419 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.782 [plugin_manager.cc:540]619419 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.793 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.818 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.847 [plugin_manager.cc:547]619419 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.430.863 [plugin_manager.cc:206]619419 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.232 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.272 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.289 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.301 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.344 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.693 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.728 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.745 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.757 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.792 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.831 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.847 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.864 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.881 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.927 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.944 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.431.955 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.431.969 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.432.323 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.361 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.381 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.394 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.432.432 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.432.778 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.812 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.830 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.842 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.432.878 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.913 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.929 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.432.942 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.432.971 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.004 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.019 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.031 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.433.047 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.433.386 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.421 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.438 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.452 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.433.489 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.433.834 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.868 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.884 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.895 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.433.931 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.966 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.983 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.433.996 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.023 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.052 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.067 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.080 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.096 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.431 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.465 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.484 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.497 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.534 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.873 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.906 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.923 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.434.935 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.434.971 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.006 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.022 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.049 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.435.063 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.092 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.107 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.118 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.435.134 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.435.471 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.505 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.521 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.533 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.435.579 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.435.924 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.958 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.975 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.435.988 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.023 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.060 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.076 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.106 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.119 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.153 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.179 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.197 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.211 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.225 [plugin_manager.cc:218]619419 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.246 [plugin_manager.cc:422]619419 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.258 [plugin_manager.cc:155]619419 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.281 [plugin_manager.cc:158]619419 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.296 [plugin_manager.cc:426]619419 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.631 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.665 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.682 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.436.694 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.436.745 [plugin_manager.cc:439]619419 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.070 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.100 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.121 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.226 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.244 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.437.262 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.439.615 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.439.656 [op_impl_registry_holder_manager.cc:216]619419 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.439.675 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.439.723 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.440.291 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.440.409 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.440.429 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.440.442 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.442.661 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.442.711 [op_impl_registry_holder_manager.cc:216]619419 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.442.730 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.442.765 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.137 [op_impl_space_registry.cc:80]619419 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.159 [op_impl_space_registry.cc:83]619419 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.185 [op_impl_space_registry.cc:80]619419 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.200 [op_impl_space_registry.cc:83]619419 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619419,python3.9):2025-04-03-09:35:44.443.213 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.258 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.443.294 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.443.381 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.443.400 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.443.411 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.444.119 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.444.148 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.444.255 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.444.278 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.444.307 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.444.896 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.444.924 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.444.965 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.445.000 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.084 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.101 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.113 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.682 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.445.708 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.794 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.810 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.445.822 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.446.388 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.446.414 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.446.455 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.446.503 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.446.586 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.446.603 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.446.615 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.447.181 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.447.208 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.447.291 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.447.309 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.447.320 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.447.882 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.447.908 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.447.944 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.447.977 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.058 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.076 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.089 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.668 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.448.708 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.796 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.813 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.448.826 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.449.390 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.449.417 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.449.453 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.449.486 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.449.519 [opp_so_manager.cc:97]619419 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.449.589 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.449.634 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.449.650 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.449.661 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.354 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.374 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.396 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.418 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.477 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.498 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.520 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.539 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.559 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.579 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.597 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.615 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.634 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.667 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.687 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.704 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.737 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.756 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.774 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.794 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.811 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.826 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.844 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.862 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.881 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.901 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.920 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.474.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.010 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.049 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.068 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.085 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.103 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.123 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.142 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.161 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.181 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.201 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.222 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.241 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.279 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.297 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.334 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.374 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.393 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.430 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.448 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.468 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.487 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.515 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.554 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.572 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.592 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.612 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.632 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.650 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.671 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.690 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.706 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.722 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.760 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.798 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.817 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.837 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.858 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.878 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.898 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.917 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.936 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.955 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.974 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.475.993 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.012 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.039 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.060 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.097 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.138 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.176 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.218 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.258 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.278 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.298 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.319 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.340 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.361 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.380 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.399 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.418 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.439 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.459 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.477 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.496 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.515 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.532 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.551 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.584 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.606 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.627 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.647 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.667 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.693 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.740 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.784 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.826 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.846 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.889 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.909 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.930 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.970 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.476.995 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.017 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.039 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.061 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.104 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.126 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.147 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.247 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.271 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.371 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.396 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.419 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.439 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.460 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.481 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.501 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.521 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.543 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.587 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.607 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.649 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.670 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.712 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.732 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.753 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.774 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.794 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.834 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.880 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.928 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.970 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.477.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.012 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.032 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.073 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.093 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.114 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.137 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.157 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.178 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.196 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.257 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.279 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.299 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.319 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.339 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.360 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.380 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.407 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.432 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.453 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.473 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.495 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.519 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.542 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.564 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.608 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.650 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.671 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.695 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.716 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.737 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.755 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.775 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.795 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.816 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.836 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.858 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.878 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.898 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.938 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.967 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.478.989 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.108 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.308 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.405 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.511 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.536 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.690 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.790 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.889 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.479.983 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.203 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.339 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.386 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.482 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.580 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.605 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.628 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.671 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.728 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.825 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.480.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.035 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.160 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.253 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.357 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.380 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.403 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.425 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.522 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.617 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.724 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.847 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.481.942 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.035 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.060 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.157 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.355 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.485 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.577 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.683 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.782 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.886 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.482.986 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.104 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.129 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.151 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.176 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.198 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.218 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.259 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.280 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.303 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.325 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.417 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.526 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.551 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.573 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.595 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.617 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.638 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.741 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.836 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.942 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.483.968 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.181 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.207 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.234 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.331 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.436 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.538 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.640 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.737 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.834 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.484.933 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.122 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.222 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.325 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.350 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.464 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.564 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.589 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.687 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.787 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.883 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.485.979 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.092 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.213 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.314 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.411 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.509 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.533 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.728 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.833 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.486.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.121 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.147 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.272 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.298 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.323 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.346 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.367 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.390 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.433 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.476 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.497 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.528 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.550 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.572 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.594 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.638 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.735 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.487.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.038 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.245 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.371 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.397 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.420 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.514 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.540 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.681 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.727 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.748 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.841 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.488.945 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.056 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.155 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.254 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.353 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.448 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.654 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.753 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.846 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.489.943 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.042 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.095 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.122 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.220 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.315 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.409 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.505 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.600 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.706 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.799 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.894 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.490.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.092 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.205 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.316 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.436 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.532 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.625 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.650 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.674 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.698 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.720 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.745 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.768 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.795 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.886 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.491.914 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.013 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.109 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.216 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.322 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.424 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.525 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.624 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.648 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.840 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.492.939 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.033 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.151 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.246 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.348 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.372 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.394 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.415 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.437 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.459 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.484 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.506 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.526 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.546 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.593 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.614 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.635 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.677 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.697 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.740 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.836 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.930 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.493.954 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.055 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.162 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.257 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.333 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.431 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.630 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.701 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.798 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.896 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.494.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.094 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.192 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.286 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.387 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.488 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.513 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.609 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.704 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.899 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.495.998 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.023 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.121 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.238 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.335 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.436 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.545 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.643 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.743 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.841 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.496.939 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.034 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.133 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.230 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.327 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.354 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.557 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.754 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.497.953 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.148 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.242 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.340 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.560 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.667 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.498.971 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.068 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.265 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.656 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.762 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.858 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.499.959 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.055 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.171 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.198 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.394 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.494 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.520 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.543 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.566 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.610 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.634 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.652 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.677 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.781 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.809 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.500.926 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.026 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.126 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.224 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.321 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.344 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.367 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.465 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.561 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.684 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.783 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.976 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.501.999 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.045 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.167 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.190 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.286 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.310 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.343 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.464 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.486 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.512 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.600 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.696 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.794 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.842 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.502.960 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.165 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.264 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.294 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.435 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.458 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.551 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.645 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.746 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.845 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.503.944 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.044 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.151 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.257 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.361 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.460 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.753 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.856 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.504.955 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.084 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.106 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.203 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.228 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.251 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.345 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.544 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.738 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.836 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.505.930 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.034 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.131 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.229 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.254 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.295 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.415 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.511 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.710 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.803 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.506.905 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.002 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.103 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.208 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.307 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.404 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.503 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.600 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.697 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.797 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.507.988 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.083 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.200 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.404 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.501 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.594 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.802 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.508.927 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.121 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.219 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.320 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.423 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.524 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.620 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.815 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.509.916 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.016 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.111 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.213 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.310 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.409 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.514 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.613 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.709 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.805 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.904 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.510.999 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.112 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.210 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.309 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.405 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.504 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.604 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.721 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.750 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.778 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.806 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.861 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.910 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.933 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.956 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.511.978 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.001 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.026 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.051 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.075 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.179 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.282 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.377 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.472 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.587 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.686 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.790 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.871 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.898 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.512.996 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.090 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.196 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.292 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.387 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.488 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.512 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.535 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.557 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.580 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.707 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.735 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.761 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.789 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.840 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.918 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.943 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.513.979 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.031 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.063 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.090 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.117 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.142 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.166 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.244 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.349 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.468 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.564 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.663 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.761 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.965 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.514.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.085 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.179 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.288 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.384 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.410 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.506 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.601 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.707 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.804 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.515.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.062 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.170 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.274 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.381 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.477 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.552 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.629 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.714 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.813 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.516.919 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.015 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.092 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.192 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.219 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.321 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.418 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.519 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.614 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.710 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.807 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.517.913 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.020 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.217 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.311 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.426 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.451 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.475 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.499 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.518 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.661 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.758 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.838 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.864 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.518.981 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.274 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.468 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.492 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.514 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.647 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.676 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.747 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.867 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.519.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.059 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.155 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.273 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.377 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.476 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.574 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.644 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.763 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.862 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.520.972 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.069 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.164 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.360 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.435 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.459 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.558 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.653 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.747 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.844 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.944 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.971 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.521.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.030 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.051 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.074 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.096 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.118 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.139 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.162 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.184 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.225 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.246 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.342 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.442 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.534 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.563 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.684 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.874 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.522.974 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.077 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.107 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.130 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.152 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.278 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.374 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.468 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.609 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.708 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.732 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.828 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.523.926 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.029 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.132 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.277 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.370 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.396 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.497 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.521 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.544 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.589 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.631 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.691 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.722 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.764 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.903 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.927 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.524.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.047 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.071 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.094 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.139 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.162 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.184 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.251 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.373 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.447 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.636 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.704 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.525.777 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.164 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.233 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.313 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.388 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.632 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.730 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.833 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.906 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.526.982 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.527.050 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.527.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.527.318 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.536.342 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.540.711 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.561.967 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.562.289 [plugin_manager.cc:554]619419 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619419,python3.9):2025-04-03-09:35:44.562.311 [plugin_manager.cc:134]619419 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.562.355 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.562.375 [plugin_manager.cc:145]619419 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.562.391 [plugin_manager.cc:557]619419 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.562.407 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.562.435 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.093 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.162 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.187 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.264 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.282 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.338 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.355 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.369 [opp_so_manager.cc:66]619419 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.385 [plugin_manager.cc:616]619419 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.397 [plugin_manager.cc:108]619419 GetOppPath:Enter get opp path schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.421 [plugin_manager.cc:117]619419 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.441 [plugin_manager.cc:624]619419 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.455 [plugin_manager.cc:206]619419 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.824 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.563.862 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.563.880 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.563.893 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.563.938 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.310 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.364 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.383 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.398 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.438 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.480 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.497 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.513 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.528 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.545 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.890 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.925 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.942 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.564.956 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.564.991 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.335 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.370 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.386 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.399 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.436 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.484 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.500 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.513 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.530 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.547 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.890 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.925 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.942 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.565.954 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.565.988 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.330 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.364 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.381 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.393 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.429 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.465 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.482 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.510 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.524 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.539 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.882 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.916 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.934 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.566.949 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.566.983 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.331 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.364 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.380 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.395 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.431 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.467 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.484 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.501 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.515 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.531 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.888 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.923 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.939 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.567.952 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.567.985 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.345 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.381 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.399 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.411 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.449 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.486 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.502 [plugin_manager.cc:1026]619419 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.568.515 [plugin_manager.cc:322]619419 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.531 [plugin_manager.cc:212]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.548 [plugin_manager.cc:215]619419 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.565 [plugin_manager.cc:218]619419 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.597 [plugin_manager.cc:422]619419 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.610 [plugin_manager.cc:155]619419 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.631 [plugin_manager.cc:158]619419 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.649 [plugin_manager.cc:426]619419 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.568.987 [plugin_manager.cc:250]619419 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.569.021 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.569.038 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.569.050 [plugin_manager.cc:1063]619419 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.088 [plugin_manager.cc:439]619419 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.418 [plugin_manager.cc:996]619419 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.447 [plugin_manager.cc:1011]619419 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.469 [plugin_manager.cc:1015]619419 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.569.523 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.628 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.659 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.569.674 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.259 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff207c02c0, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.296 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.317 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.336 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.360 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.658 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff207c4978, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.683 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.704 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.723 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.742 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.571.997 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.572.025 [op_impl_registry_holder_manager.cc:216]619419 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.042 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.077 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.335 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.572.389 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.516 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.535 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.572.552 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.040 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.073 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.093 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.113 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.134 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.417 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff207719b0, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.441 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.461 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.479 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.497 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.774 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff20774da0, registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.797 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.815 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.833 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.574.850 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.096 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.123 [op_impl_registry_holder_manager.cc:216]619419 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.152 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.193 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.497 [op_impl_space_registry.cc:84]619419 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.517 [op_impl_space_registry.cc:85]619419 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.531 [op_impl_space_registry.cc:86]619419 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.546 [op_impl_space_registry.cc:87]619419 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.558 [op_impl_space_registry.cc:93]619419 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.579 [op_impl_space_registry.cc:84]619419 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.594 [op_impl_space_registry.cc:85]619419 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.605 [op_impl_space_registry.cc:86]619419 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.618 [op_impl_space_registry.cc:87]619419 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.632 [op_impl_space_registry.cc:93]619419 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.645 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.696 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.735 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.575.774 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.872 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.901 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.575.913 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.576.456 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.576.483 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.576.530 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.576.626 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.576.643 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.576.655 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.577.274 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.577.300 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.577.341 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.577.379 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.577.415 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.577.510 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.577.526 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.577.537 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.577.988 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.011 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.051 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.145 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.163 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.176 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.735 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.760 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.805 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.842 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.578.878 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.971 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.578.988 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.579.000 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.579.424 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.579.460 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.579.504 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.579.596 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.579.612 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.579.627 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.580.203 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.233 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.277 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.314 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.350 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.580.443 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.580.459 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.580.470 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.580.893 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.915 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.580.973 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.065 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.082 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.093 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.648 [op_impl_space_registry.cc:240]619419 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.673 [opp_so_manager.cc:48]619419 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.713 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.750 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.787 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.813 [opp_so_manager.cc:126]619419 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.581.844 [opp_so_manager.cc:123]619419 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.926 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.943 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.581.987 [plugin_manager.cc:1126]619419 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.582.004 [plugin_manager.cc:1130]619419 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.582.016 [opp_so_manager.cc:45]619419 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.594.591 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.595.358 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.595.409 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.595.439 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.598.074 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619419,python3.9):2025-04-03-09:35:44.612.255 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:44.612.253 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:44.612.290 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:44.612.298 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.309 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.314 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.337 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.364 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.612.369 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.612.382 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.612.394 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.612.396 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.419 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.612.419 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.694.327 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.694.802 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.694.884 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.694.959 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.028 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.092 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.190 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.213 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.276 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.298 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.318 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.336 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.445 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.516 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.585 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.648 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.807 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.870 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.897 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.919 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.695.984 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.116 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.200 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.269 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.334 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.398 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.467 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.530 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.674 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.740 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.803 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.696.936 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.697.004 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.697.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.702.433 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.702.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.702.672 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.702.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.065 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.168 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.268 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.423 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.678 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.819 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.703.922 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.035 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.178 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.289 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.403 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.428 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.589 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.702 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.839 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.704.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.081 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.189 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.304 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.451 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.559 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.667 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.776 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.705.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.006 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.118 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.431 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.541 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.657 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.685 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.706.891 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.007 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.145 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.443 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffeea0ac360, registered count:5 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.473 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.495 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.526 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.551 [op_check.cc:71]619419 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.574 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.695 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.803 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.707.998 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.168 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.278 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.304 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.407 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.433 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.536 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.643 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.667 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.771 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.877 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.708.980 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.090 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.281 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.306 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.412 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.513 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.696 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.804 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.709.911 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.132 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.326 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.480 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.617 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.759 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.869 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.710.980 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.091 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.191 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.553 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.712 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.737 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffeea136e08, registered count:7 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.711.915 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffeea13f298, registered count:7 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.026 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.130 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.279 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.397 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.504 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.615 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.821 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.712.931 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.047 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.177 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.283 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.391 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.497 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.600 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.713 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.817 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.713.929 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.065 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.093 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.119 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.142 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.288 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.396 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.501 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.613 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.724 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.830 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.854 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.883 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.714.994 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.098 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.200 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.389 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.612 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.725 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.837 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.715.946 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.058 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.084 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.201 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.313 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.341 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.608 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.715 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.825 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.716.935 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.040 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.146 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.251 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.357 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.543 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.649 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.717.759 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.012 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.059 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.083 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.214 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.346 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.463 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.567 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.675 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.720.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.114 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.141 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.253 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.278 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.598 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.626 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.735 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.845 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.721.947 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.058 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.278 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.554 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.698 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.803 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.722.923 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.723.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.723.666 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.724.462 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.724.780 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.725.224 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.725.806 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.725.926 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.726.041 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.726.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.726.285 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.726.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.728.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.729.512 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.729.674 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.729.831 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.730.785 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.730.938 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.080 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.637 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.750 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.864 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.731.970 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.082 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.310 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.418 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.530 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.644 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.732.916 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.022 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.201 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.305 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.416 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.523 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.676 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.849 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.733.964 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.073 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.099 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.280 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.386 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.490 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.655 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.767 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.734.915 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.243 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.270 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.375 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.401 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.509 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.615 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.651 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.758 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.867 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.735.973 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.172 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.284 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.472 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.620 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.791 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.736.957 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.128 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.426 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.533 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.637 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.744 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.771 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.737.949 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.055 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.158 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.262 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.368 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.484 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.612 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.738.945 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.739.944 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.742.352 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.742.547 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.742.751 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.743.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.743.230 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.743.376 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.338 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.466 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.616 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.761 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.744.880 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.024 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.174 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.311 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.435 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.540 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.630 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.941 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.968 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.745.991 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.014 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.039 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.073 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.096 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.118 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.139 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.160 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.182 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.205 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.226 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.248 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.268 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.290 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.332 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.352 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.375 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.396 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.418 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.440 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.482 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.506 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.530 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.551 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.572 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.593 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.622 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.645 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.672 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.696 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.717 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.738 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.759 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.781 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.802 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.822 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.840 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.860 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.880 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.902 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.924 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.946 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.968 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.746.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.013 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.035 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.055 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.078 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.099 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.121 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.144 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.176 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.322 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.680 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.842 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.868 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.891 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.912 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.747.935 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.072 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.204 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.317 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.422 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.527 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.632 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.781 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.748.892 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.051 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.157 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.271 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.321 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.424 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.450 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.474 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.584 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.735 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.749.893 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.031 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.137 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.326 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.432 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.607 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.714 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.829 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.854 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.750.877 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.019 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.167 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.463 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.779 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.751.885 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.040 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.377 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.500 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.612 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.718 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.827 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.752.934 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.091 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.344 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.449 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.669 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.753.778 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.191 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.603 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.727 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.844 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.871 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.754.975 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.081 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.187 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.297 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.406 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.516 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.622 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.725 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.881 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.755.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.104 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.301 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.409 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.539 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.645 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.822 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.756.988 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.098 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.218 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.328 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.430 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.533 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.747 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.892 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.757.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.758.146 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.758.388 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.758.568 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.758.729 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.758.897 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.099 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.299 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.496 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.521 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.542 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.562 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.586 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.619 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.642 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.759.665 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.760.398 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.760.790 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.761.180 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.761.376 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.761.572 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.761.960 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.762.077 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.762.237 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.764.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.766.389 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.766.511 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.766.622 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.766.726 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.766.997 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.142 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.320 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.648 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.680 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.705 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.728 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.750 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.914 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.767.953 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.109 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.281 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.446 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.611 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.784 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.768.960 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.118 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.282 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.439 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.602 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.745 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.771 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.769.922 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.095 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.122 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.223 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.338 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.365 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.480 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.505 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.727 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.847 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.770.966 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.771.064 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.771.169 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.771.347 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.771.534 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.771.891 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.772.260 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.772.450 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.772.777 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.772.961 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.146 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.259 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.372 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.512 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.542 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.568 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.773.592 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.774.644 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.058 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.218 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.503 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.532 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.676 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.832 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.775.979 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.006 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.152 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.320 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.460 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.618 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.776.761 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.777.141 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.777.474 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.777.701 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.777.879 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.779.127 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.779.538 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.070 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.256 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.411 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.526 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.670 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.824 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.853 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.956 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.780.982 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.781.296 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.781.414 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.781.556 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.781.725 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.781.836 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.130 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.308 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.455 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.565 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.673 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.818 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.782.990 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.163 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.191 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.341 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.368 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.513 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.783.686 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.785.135 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.790.665 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.790.712 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.793.199 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.793.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.793.742 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.057 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.259 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.462 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.621 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.788 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.794.969 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.795.101 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.795.185 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.795.422 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.795.536 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.795.684 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.138 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.312 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.461 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.491 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.608 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.721 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.796.895 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.074 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.257 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.400 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.508 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.615 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.640 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.797.827 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.001 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.149 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.259 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.414 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.453 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.570 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.679 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.703 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.727 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.798.866 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.052 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.140 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.363 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.547 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.577 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.604 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.781 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.799.810 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.404 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.497 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.523 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.590 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.664 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.686 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.756 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.907 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.802.928 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.037 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.058 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.078 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.111 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.268 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.803.288 [tuning_tiling_registry.cc:30]619419 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.804.393 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffeeaa7b0b0, registered count:7 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.804.428 [op_check.cc:24]619419 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffeeaa7b310, registered count:5 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.804.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.805.300 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.807.119 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.807.282 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.807.365 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.807.716 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.808.143 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.837.518 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.837.639 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.837.812 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.837.889 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.838.027 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.838.164 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.838.814 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.838.896 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.839.141 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.839.223 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.839.362 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.839.588 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.839.762 [op_impl_registry.cc:122]619419 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619419,python3.9):2025-04-03-09:35:44.840.433 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.846.159 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619419,python3.9):2025-04-03-09:35:44.848.101 [op_impl_registry_holder_manager.cc:208]619419 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.923.515 [op_impl_space_registry.cc:109]619419 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.924.843 [op_impl_space_registry.cc:84]619419 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.924.876 [op_impl_space_registry.cc:85]619419 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.924.888 [op_impl_space_registry.cc:86]619419 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.924.903 [op_impl_space_registry.cc:87]619419 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.924.916 [op_impl_space_registry.cc:93]619419 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619419,python3.9):2025-04-03-09:35:44.925.923 [op_impl_space_registry.cc:273]619419 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.405 [plugin_manager.cc:564]619419 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.433 [plugin_manager.cc:134]619419 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.926.500 [file_utils.cc:47]619419 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.926.519 [plugin_manager.cc:145]619419 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619419,python3.9):2025-04-03-09:35:44.926.532 [plugin_manager.cc:567]619419 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.607 [mem_manager.cc:31]619419 Initialize:MemManager Initialize. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.642 [rts_caching_mem_allocator.cc:131]619419 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.661 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.677 [graph_mem_allocator.cc:17]619419 Initialize:MemoryAllocator::Initialize. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.695 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.709 [graph_mem_allocator.cc:17]619419 Initialize:MemoryAllocator::Initialize. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.722 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.737 [caching_allocator.cc:391]619419 FreeBlocks:Free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.754 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.767 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.806 [caching_allocator.cc:391]619419 FreeBlocks:Free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.816 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.830 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.845 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.857 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.871 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.884 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.899 [session_scope_mem_allocator.cc:73]619419 FreeAllMemory:Free all memory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.912 [mem_manager.cc:102]619419 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.926.926 [session_scope_mem_allocator.cc:73]619419 FreeAllMemory:Free all memory [INFO] GE(619419,python3.9):2025-04-03-09:35:44.927.008 [dump_manager.cc:152]619419 Init:exception dump is not enabled [INFO] GE(619419,python3.9):2025-04-03-09:35:44.927.054 [ge_executor.cc:315]619419 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.076 [acl.cpp:321]619419 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.092 [dump.cpp:841]619419 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.259 [dump.cpp:759]619419 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.274 [dump.cpp:594]619419 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.285 [dump.cpp:224]619419 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.328 [dump.cpp:703]619419 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.347 [dump.cpp:771]619419 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.927.363 [dump.cpp:703]619419 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.427 [adx_datadump_server.cpp:77][tid:619419] start to do dump init [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.450 [adx_dump_record.cpp:210][tid:619419] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.466 [adx_dump_record.cpp:214][tid:619419] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.701 [adx_server_manager.cpp:149][tid:619732] server manager add component (10) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.719 [adx_dump_record.cpp:627][tid:619733] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.729 [adx_server_manager.cpp:165][tid:619732] server manager components init successfully [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.874 [adx_server_manager.cpp:203][tid:619734] Run Server(0) Process [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.929 [adx_server_manager.cpp:369][tid:619734] device up 0 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.927.950 [hdc_comm_opt.cpp:59][tid:619734] open device[0] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.073 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.101 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.119 [hdc_api.cpp:119][tid:619734] logDevId 0 create HDC server successfully [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.177 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.193 [adx_server_manager.cpp:369][tid:619734] device up 1 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.204 [hdc_comm_opt.cpp:59][tid:619734] open device[1] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.341 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.357 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.370 [hdc_api.cpp:119][tid:619734] logDevId 1 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.384 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.397 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.411 [adx_server_manager.cpp:369][tid:619734] device up 2 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.421 [hdc_comm_opt.cpp:59][tid:619734] open device[2] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.495 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.511 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.527 [hdc_api.cpp:119][tid:619734] logDevId 2 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.538 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.552 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.565 [adx_server_manager.cpp:369][tid:619734] device up 3 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.576 [hdc_comm_opt.cpp:59][tid:619734] open device[3] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.709 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.725 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.741 [hdc_api.cpp:119][tid:619734] logDevId 3 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.751 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.765 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.778 [adx_server_manager.cpp:369][tid:619734] device up 4 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.788 [hdc_comm_opt.cpp:59][tid:619734] open device[4] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.921 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.928.944 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.956 [hdc_api.cpp:119][tid:619734] logDevId 4 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.972 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.983 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.928.994 [adx_server_manager.cpp:369][tid:619734] device up 5 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.006 [hdc_comm_opt.cpp:59][tid:619734] open device[5] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.138 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.154 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.169 [hdc_api.cpp:119][tid:619734] logDevId 5 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.180 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.194 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.207 [adx_server_manager.cpp:369][tid:619734] device up 6 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.217 [hdc_comm_opt.cpp:59][tid:619734] open device[6] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.351 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.367 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.379 [hdc_api.cpp:119][tid:619734] logDevId 6 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.393 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.405 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.418 [adx_server_manager.cpp:369][tid:619734] device up 7 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.428 [hdc_comm_opt.cpp:59][tid:619734] open device[7] server[13] [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.561 [drv_log_user.c:623][ascend][curpid:619419,619734][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.929.577 [hdc_server.c:363][ascend][curpid:619419,619734][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.592 [hdc_api.cpp:119][tid:619734] logDevId 7 create HDC server successfully [WARNING] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.603 [adx_hdc_epoll.cpp:34][tid:619734] hdc epoll has been created. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.929.617 [adx_server_manager.cpp:111][tid:619734] create server info [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.930.811 [adx_server_manager.cpp:457][tid:619419] The server is initialized after waiting 3 times. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.930.833 [ge_executor.cc:1349]619419 SetDump:Start to set dump config [INFO] GE(619419,python3.9):2025-04-03-09:35:44.930.853 [dump_manager.cc:201]619419 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.930.891 [dump_manager.cc:261]619419 SetExceptionDump:Enable brief exception dump. [INFO] GE(619419,python3.9):2025-04-03-09:35:44.930.934 [ge_executor.cc:1355]619419 SetDump:Set dump config successfully [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.930.971 [dump_manager.cpp:146][tid:619419] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:44.932.405 [dump_manager.cpp:301][tid:619419] Register exception callback, registered: 1 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.431 [dump.cpp:825]619419 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.452 [acl.cpp:327]619419 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.466 [acl.cpp:329]619419 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.481 [acl_resource_manager.cpp:248]619419 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.581 [acl_resource_manager.cpp:275]619419 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.598 [acl_resource_manager.cpp:278]619419 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.610 [acl.cpp:335]619419 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.623 [acl.cpp:336]619419 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.688 [json_parser.cpp:260]619419 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.705 [acl.cpp:343]619419 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:44.932.726 [prof_atls_plugin.cpp:210] (tid:619419) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.742 [acl.cpp:351]619419 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.754 [profiling.cpp:141]619419 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.769 [profiling.cpp:135]619419 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.828 [profiling.cpp:167]619419 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619419.json], errorCode = 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.845 [profiling.cpp:172]619419 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619419.json] is empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.856 [profiling.cpp:176]619419 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.868 [profiling.cpp:179]619419 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.932.882 [profiling.cpp:92]619419 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:44.933.284 [ai_drv_dev_api.cpp:316] >>> (tid:619419) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:44.933.398 [msprof_callback_impl.cpp:132] >>> (tid:619419) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.416 [profiling.cpp:122]619419 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.428 [profiling.cpp:186]619419 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.458 [hash_utils.cpp:108]619419 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619419.json [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.496 [hash_utils.cpp:128]619419 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619419.json, hash: 3917491750093904922 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.512 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.549 [acl.cpp:280]619419 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.563 [acl.cpp:385]619419 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.933.584 [runtime.cc:5394] 619419 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.933.603 [api_impl.cc:7407] 619419 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.933.640 [api_impl.cc:4211] 619419 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.933.659 [acl.cpp:420]619419 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.933.851 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.933.983 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.934.047 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.934.073 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.939443] [info] [619419] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist mki_log delete old file:/home/jenkins/atb/log/atb_1952742_20250328090704.log [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:44.940.144 [prof_atls_plugin.cpp:210] (tid:619419) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.941.022 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.941.050 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.941.129 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 0, rank size = 8. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.941.149 [device.cpp:143]619419 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.178 [api_impl.cc:2999] 619419 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.196 [device.cc:306] 619419 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.214 [runtime.cc:3393] 619419 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.233 [runtime.cc:3560] 619419 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.260 [api_impl.cc:2987] 619419 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.278 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.299 [api_impl.cc:2992] 619419 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.319 [api_impl.cc:3025] 619419 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.941.336 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.941.352 [acl.cpp:121]619419 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.941.374 [device.cpp:154]619419 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.941.435 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.941.455 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is _npu_log, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.479 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.495 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.941.513 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.941.550 [queue_client.c:151][ascend][curpid:619419,619419][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.941.732 [queue_client.c:305][ascend][curpid:619419,619419][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=0; vfid=0; hostpid=619419; master_pid=619419; proc_type=0; devpid=319). [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.945.707 [queue_client.c:362][ascend][curpid:619419,619419][drv][queuemng][QueueInit]QueueInit device. (devid=0; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.945.902 [npu_driver.cc:4114] 619419 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.945.925 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.945.942 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=_npu_log. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.946.244 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.264 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.390 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.425 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.946.449 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.460 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.472 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.487 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.495 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.510 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.523 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.568 [npu_driver.cc:4087] 619419 MemQueueInit: repeated init [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.586 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.601 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=ms_tensor_dump. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.946.805 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.824 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.929 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.946.959 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.946.978 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.946.981 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.002 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.000 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.024 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.040 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.051 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.089 [npu_driver.cc:4087] 619419 MemQueueInit: repeated init [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.105 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.121 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=ms_tensor_summary. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.947.277 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.296 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.471 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.947.504 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.506 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.524 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.539 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.543 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.553 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.560 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.573 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.609 [npu_driver.cc:4087] 619419 MemQueueInit: repeated init [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.625 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.636 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=ms_image_summary. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.947.799 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.817 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.914 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.942 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.947.942 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.960 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.947.964 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.977 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.984 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.947.998 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.010 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.048 [npu_driver.cc:4087] 619419 MemQueueInit: repeated init [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.063 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.088 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=ms_scalar_summary. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.948.238 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.257 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.373 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.948.404 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.409 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.421 [tensor_data_transfer.cpp:898]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.434 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.442 [api_impl.cc:6077] 619419 MemQueueInit: Start to init queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.448 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.456 [npu_driver.cc:4076] 619419 MemQueueInit: init mem queue, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.468 [npu_driver.cc:4026] 619419 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.504 [npu_driver.cc:4087] 619419 MemQueueInit: repeated init [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.520 [api_impl.cc:6040] 619419 MemQueueCreate: Start to create queue on devId 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.536 [npu_driver.cc:3931] 619419 MemQueueCreate: create mem queue, deviceId=0, name=ms_histogram_summary. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.948.688 [queue_interface.c:182][ascend][curpid:619419,619419][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.706 [tensor_data_transfer.cpp:940]619419 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 0, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.874 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.948.900 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.948.914 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.921 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.948.940 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.949.071 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.949.115 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.134 [plugin_manager.cc:25] [619419]hcom ops plugin init start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.171 [plugin_manager.cc:38] [619419]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.194 [plugin_manager.cc:42] [619419]hcom running normal mode. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.238 [hcom_plugin.cc:50] [619419]Initialize start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.253 [hcom_plugin.cc:52] [619419]initializeCount_ : 1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.299 [hcom_plugin.cc:60] [619419]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.325 [hcom_plugin.cc:63] [619419]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.342 [hcom_plugin.cc:329] [619419]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.359 [hcom_plugin.cc:281] [619419]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.949.383 [hcom_plugin.cc:217] [619419]option profiling mode is not found. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.401 [hcom_plugin.cc:164] [619419][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.418 [hcom_plugin.cc:180] [619419][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.434 [hcom_plugin.cc:244] [619419]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.562 [hcom_plugin.cc:82] [619419]hccl ops plugin init success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.579 [plugin_manager.cc:48] [619419]hcom ops plugin init end. ret[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.593 [hcom_plugin.cc:126] [619419]get hccl kernel info store start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.620 [hcom_plugin.cc:132] [619419]get hccl kernel info store finished. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.949.649 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.949.675 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.689 [hcom_executor.cc:29] [619419]Hcom Excutor Initialize start. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.949.748 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.768 [adapter_rts.cc:234] [619419][hrtGetDevice]deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.949.812 [api_c_soc.cc:70] 619419 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.852 [hcom_common.cc:85] [619419][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.882 [hcom_common.cc:88] [619419][HcomGetCurHcomCtx] is no Used deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.908 [hcom.cc:486] [619419]Get Hcom Init Status: [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.924 [hcom_common.cc:85] [619419][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.949.936 [hcom_rpc.cc:1019] [619419]Get Hcom Init Status: [0] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.949.961 [hcom_base_funs.cc:36] [619419][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.950.002 [hcom_executor.cc:121] [619419][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.058 [api_impl.cc:1120] 619419 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.086 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.150 [stream.cc:2246] 619419 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.219 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.237 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.747 [npu_driver.cc:6125] 619419 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.796 [npu_driver.cc:6137] 619419 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=499, streamId=4, logicCq=499, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.825 [npu_driver.cc:6189] 619419 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=4, logicCq=499. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.845 [npu_driver.cc:5916] 619419 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d85e3000, addr=dfffd85e3000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.862 [stream.cc:653] 619419 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.950.936 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.950.955 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.951.295 [stream.cc:291] 619419 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.951.881 [stream.cc:688] 619419 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=0, streamResId=65535 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.951.902 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.951.917 [acl_resource_manager.cpp:816]619419 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.951.931 [api_impl.cc:1162] 619419 StreamCreate: Succ, flags=1536, stream_id=4, context=aaaac1adc2f0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.951.972 [stream.cc:73] [619419]Construct stream by stream type success, ptr[0xaaaaccee9b80] ctx[0xaaaac1adc2f0], stream id[4], cqId[4], logicCqid[499] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.012 [api_impl.cc:1120] 619419 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.039 [npu_driver.cc:404] 619419 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.069 [stream.cc:2246] 619419 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.108 [stream_sqcq_manage.cc:147] 619419 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.124 [npu_driver.cc:882] 619419 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.361 [npu_driver.cc:6125] 619419 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.411 [npu_driver.cc:6137] 619419 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=500, streamId=5, logicCq=500, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.442 [npu_driver.cc:6189] 619419 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=5, logicCq=500. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.460 [npu_driver.cc:5916] 619419 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d85c2000, addr=dfffd85c2000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.476 [stream.cc:653] 619419 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619419,python3.9):2025-04-03-09:35:44.952.521 [devmm_svm.c:1972][ascend][curpid:619419,619419][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.540 [npu_driver.cc:6080] 619419 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.952.837 [stream.cc:291] 619419 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.468 [stream.cc:688] 619419 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=0, streamResId=65535 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.953.489 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.953.500 [acl_resource_manager.cpp:816]619419 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.515 [api_impl.cc:1162] 619419 StreamCreate: Succ, flags=1536, stream_id=5, context=aaaac1adc2f0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.953.537 [stream.cc:73] [619419]Construct stream by stream type success, ptr[0xaaaaccf65be0] ctx[0xaaaac1adc2f0], stream id[5], cqId[5], logicCqid[500] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.953.677 [sal.cc:619] [619747]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.953.715 [hcom_executor.cc:587] [619747]MessageProcessThreadLoop start queType:[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.741 [api_impl.cc:2999] 619747 SetDevice: device_id=0, deviceMode=0. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.953.736 [sal.cc:619] [619748]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.759 [device.cc:306] 619747 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.768 [api_impl.cc:2999] 619749 SetDevice: device_id=0, deviceMode=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.953.773 [hcom_executor.cc:34] [619419][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.953.779 [hcom_executor.cc:587] [619748]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.785 [runtime.cc:3393] 619747 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.798 [device.cc:306] 619749 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.819 [runtime.cc:3560] 619747 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.812 [api_impl.cc:2999] 619750 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.819 [api_impl.cc:2999] 619748 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.823 [runtime.cc:3393] 619749 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.953.826 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.833 [api_impl.cc:2987] 619747 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.844 [device.cc:306] 619750 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.858 [runtime.cc:3560] 619749 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.953.884 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.881 [runtime.cc:5239] 619747 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.892 [runtime.cc:3393] 619750 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.893 [api_impl.cc:2987] 619749 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.895 [device.cc:306] 619748 Init: device mode has been already set, mode=0. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.953.904 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.908 [api_impl.cc:2992] 619747 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.913 [runtime.cc:3560] 619750 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.918 [runtime.cc:5239] 619749 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.925 [runtime.cc:3393] 619748 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.953.943 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.953.944 [awatchdog_core.c:175](tid:619747) create watchdog for id : 65543, tid : 619747, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.944 [api_impl.cc:2987] 619750 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.952 [api_impl.cc:2992] 619749 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.955 [runtime.cc:3560] 619748 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.970 [api_impl.cc:3025] 619747 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.953.973 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.975 [runtime.cc:5239] 619750 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.953.987 [api_impl.cc:2987] 619748 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.953.988 [awatchdog_core.c:175](tid:619749) create watchdog for id : 65543, tid : 619749, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.006 [api_impl.cc:2992] 619750 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.008 [runtime.cc:5239] 619748 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.015 [api_impl.cc:3025] 619749 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.023 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1099.27 msec. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.954.022 [awatchdog_core.c:175](tid:619750) create watchdog for id : 65543, tid : 619750, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.027 [api_impl.cc:2992] 619748 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.086 [api_impl.cc:3025] 619750 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.094 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.954.095 [awatchdog_core.c:175](tid:619748) create watchdog for id : 65543, tid : 619748, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.110 [api_impl.cc:3025] 619748 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [WARNING] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.117 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.179 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.019 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.228 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.275 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.299 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.095 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.417 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:44.954.445 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.954.517 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.954.553 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.954.595 [device.cpp:143]619751 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.625 [api_impl.cc:2999] 619751 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.641 [device.cc:306] 619751 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.658 [runtime.cc:3393] 619751 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.685 [runtime.cc:3560] 619751 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.697 [api_impl.cc:2987] 619751 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.712 [runtime.cc:5239] 619751 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.730 [api_impl.cc:2992] 619751 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.954.748 [awatchdog_core.c:175](tid:619751) create watchdog for id : 65543, tid : 619751, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.763 [api_impl.cc:3025] 619751 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.954.775 [acl.cpp:115]619751 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.954.789 [acl.cpp:121]619751 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.954.802 [device.cpp:154]619751 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.954.834 [runtime.cc:5239] 619751 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.852 [adapter_rts.cc:198] [619751][hrtGetDeviceRefresh]deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.869 [op_base.cc:802] [619751]Entry-HcclGetRootInfo:rootInfo[0xaaaacd6d40c0], deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.890 [externalinput.cc:2065] [619751]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.911 [externalinput.cc:490] [619751]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.927 [externalinput.cc:429] [619751]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.943 [externalinput.cc:543] [619751]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.961 [externalinput.cc:622] [619751]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.979 [externalinput.cc:714] [619751]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.954.996 [externalinput.cc:821] [619751]HCCL_IF_IP is not set [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.012 [externalinput.cc:877] [619751]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.028 [externalinput.cc:844] [619751]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.040 [externalinput.cc:728] [619751]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.054 [externalinput.cc:758] [619751]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.066 [externalinput.cc:806] [619751]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.081 [externalinput.cc:1508] [619751]HCCL_ALGO is not set [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.094 [externalinput.cc:1671] [619751][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.111 [externalinput.cc:462] [619751]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.127 [externalinput.cc:1519] [619751]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.143 [externalinput.cc:1554] [619751]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.955.231 [api_c_soc.cc:70] 619751 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.250 [externalinput.cc:1599] [619751]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.271 [externalinput.cc:1633] [619751]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.286 [externalinput.cc:1778] [619751]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.298 [externalinput.cc:1807] [619751]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.312 [externalinput.cc:515] [619751]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.327 [externalinput.cc:1841] [619751]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.340 [externalinput.cc:1929] [619751]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.355 [externalinput.cc:1949] [619751]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.371 [externalinput.cc:1969] [619751]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.392 [externalinput.cc:2005] [619751]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.409 [externalinput.cc:2036] [619751]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.427 [externalinput.cc:2209] [619751]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.442 [externalinput.cc:2178] [619751][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.458 [externalinput.cc:889] [619751]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.470 [externalinput.cc:914] [619751]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.485 [externalinput.cc:2240] [619751]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.497 [externalinput.cc:2308] [619751]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.514 [externalinput.cc:2339] [619751]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.530 [externalinput.cc:2296] [619751]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.546 [externalinput.cc:407] [619751]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.955.565 [externalinput.cc:672] [619751][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.955.610 [env_config.cc:127] [619751]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:44.955.630 [env_config.cc:131] [619751]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.648 [env_config.cc:302] [619751]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.664 [env_config.cc:302] [619751]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.955.697 [api_impl.cc:3100] 619751 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.955.715 [api_impl.cc:3111] 619751 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.955.736 [api_impl.cc:3130] 619751 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.749 [topoinfo_detect.cc:142] [619751][Setup][hcclIfBasePort]deviceLogicID_[0], devicePhysicID_[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.958 [network_manager.cc:69] [619751]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.955.990 [network_manager.cc:169] [619751]NetworkManager: devicePhyId[0], devicePhyId_[0] deviceLogicId_[0], hasBackup[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.023 [ra_host.c:1594]tid:619751,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.956.048 [adapter_hccp.cc:1450] [619751]set host socket whitelist status[0] success. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.088 [ra_host.c:205]tid:619751,ra_init(205) : Input parameters: phy_id[0], nic_position:[0] hdc_type:[6] [INFO] ROCE(619419,python3.9):2025-04-03-09:35:44.956.144 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.184 [ra_peer.c:776]tid:619751,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[0] start [INFO] ROCE(619419,python3.9):2025-04-03-09:35:44.956.254 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619419,python3.9):2025-04-03-09:35:44.956.321 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.347 [rs.c:327]tid:619751,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619419,python3.9):2025-04-03-09:35:44.956.518 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619419,python3.9):2025-04-03-09:35:44.956.542 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619419,python3.9):2025-04-03-09:35:44.956.558 [rs_ssl.c:1513]tid:619751,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.575 [rs_ssl.c:1529]tid:619751,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.612 [rs_epoll.c:692]tid:619751,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.709 [rs_epoll.c:487]tid:619752,rs_epoll_handle(487) : thread begin! thread_id:281469423054752, pid:619419, ppid:619411 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.806 [rs_epoll.c:713]tid:619751,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.809 [rs_epoll.c:415]tid:619752,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.813 [rs_epoll.c:623]tid:619753,rs_connect_handle(623) : thread begin! thread_id:281468869799840, pid:619419, ppid:619411 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.826 [rs.c:366]tid:619751,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.236000] ms [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.832 [rs_epoll.c:511]tid:619752,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.956.959 [rs_epoll.c:641]tid:619753,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.598 [rs.c:471]tid:619751,rs_init(471) : rs init success, chip_id[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.621 [ra_peer.c:813]tid:619751,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[0] succ [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.958.639 [adapter_hccp.cc:583] [619751]init ra success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.958.655 [network_manager.cc:222] [619751]NetworkManager nicDeploy[0] deviceLogicId_[0] devicePhyId_[0] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.692 [ra_host.c:1764]tid:619751,ra_get_ifnum(1764) : Input parameters: phy_id[0], nic_position:[0], is_all:[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.714 [ra_peer.c:859]tid:619751,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[0] start [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.739 [ra_peer.c:867]tid:619751,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[0] succ [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.958.755 [adapter_hccp.cc:1378] [619751][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.774 [ra_host.c:1814]tid:619751,ra_get_ifaddrs(1814) : Input parameters: phy_id[0], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.790 [ra_peer.c:874]tid:619751,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[0] start [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.805 [rs_socket.c:2743]tid:619751,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.821 [rs_socket.c:2743]tid:619751,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.832 [rs_socket.c:2743]tid:619751,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.848 [rs_socket.c:2749]tid:619751,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.859 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.872 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.886 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.896 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.908 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.922 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.933 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.945 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.959 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.969 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.981 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.958.995 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.005 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.017 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.031 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.042 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.054 [rs_socket.c:2749]tid:619751,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.068 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.084 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.097 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.111 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.121 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.133 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.145 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.158 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.172 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.182 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.194 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.208 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.218 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.230 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.243 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.254 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.265 [rs_socket.c:2749]tid:619751,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.280 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.290 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.302 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.316 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.326 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.338 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.351 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.362 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.374 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.387 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.398 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.409 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.423 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.434 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.461 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.472 [rs_socket.c:2751]tid:619751,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.487 [ra_peer.c:882]tid:619751,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[0] succ [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.514 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.532 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.549 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.566 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.584 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.600 [adapter_hccp.cc:1407] [619751][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.628 [sal.cc:365] [619751]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.643 [sal.cc:432] [619751]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.669 [network_manager.cc:69] [619751]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.703 [ra_host.c:340]tid:619751,ra_socket_init_v1(340) : socket init:mode=0 phy_id=0 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.723 [adapter_hccp.cc:818] [619751]socket init v1 success, socketHandle[0xfffe9808cb40] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.738 [network_manager.cc:1218] [619751]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.749 [network_manager.cc:1257] [619751]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe9808cb40] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.782 [network_manager.cc:69] [619751]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.959.803 [network_manager.cc:729] [619751][Start][HostNetAndListen]trying to listen on ip[192.168.0.44%enp67s0f5] port[60008]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.822 [ra_host.c:875]tid:619751,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], port[60008] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.851 [rs_socket.c:975]tid:619751,rs_socket_init_listen(975) : listen [0] IP 0xffff2c00a8c0, ret_vnic -1 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.871 [rs_socket.c:98]tid:619751,rs_find_listen_node(98) : listen node for IP(192.168.0.44), server_port(60008) is not listen! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.888 [rs_socket.c:98]tid:619751,rs_find_listen_node(98) : listen node for IP(192.168.0.44), server_port(60008) is not listen! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.905 [rs_socket.c:163]tid:619751,rs_listen_node_alloc(163) : create listen node for IP(192.168.0.44)! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.937 [rs_socket.c:909]tid:619751,rs_socket_listen_bind_listen(909) : listen state:1, then bind for (IP 192.168.0.44 : port 60008) [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.953 [rs_socket.c:912]tid:619751,rs_socket_listen_bind_listen(912) : socket bind: family 2, addr 192.168.0.44, port 60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.965 [rs_socket.c:918]tid:619751,rs_socket_listen_bind_listen(918) : socket bind: family 2, port 26858, addr 0x2c00a8c0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:44.959.992 [rs_socket.c:949]tid:619751,rs_socket_listen_bind_listen(949) : IP 192.168.0.44 : port 60008 begin listen, fd:53 ! [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.021 [network_manager.cc:741] [619751][Start][HostNetAndListen]Listen on ip[192.168.0.44%enp67s0f5], port[60008] sucess, devPhyId[0], devLogicId[0], isAutoPort[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.039 [network_manager.cc:744] [619751]host ip[192.168.0.44%enp67s0f5] port[60008] refcount is [1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.052 [network_manager.cc:749] [619751]HostNet, ip[192.168.0.44%enp67s0f5] port[60008] socket init OK [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.065 [network_manager.cc:69] [619751]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.083 [hccl_socket.cc:516] [619751][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.100 [hccl_socket.cc:106] [619751][HcclSocket][Listen] device[0] listens on ip[192.168.0.44%enp67s0f5] port[60008] success, socketType[2]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.115 [topoinfo_detect.cc:490] [619751]topo info exchange server start with host ip[192.168.0.44%enp67s0f5] and port[60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.135 [topoinfo_detect.cc:204] [619751]rootInfo: ip[192.168.0.44] port[60008] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.245 [topoinfo_detect.cc:175] [619751]setup topo exchange server complete, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.271 [op_base.cc:128] [619751][HcclGetDeviceId] deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.280 [api_impl.cc:2999] 619754 SetDevice: device_id=0, deviceMode=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.289 [op_base.cc:144] [619751][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.301 [device.cc:306] 619754 Init: device mode has been already set, mode=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.310 [op_base.cc:829] [619751][HCCL_TRACE]HcclGetRootInfo success, take time [5478]us, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.315 [runtime.cc:3393] 619754 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.336 [runtime.cc:3560] 619754 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.350 [api_impl.cc:2987] 619754 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.960.351 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 5.743 msec. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.365 [runtime.cc:5239] 619754 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.380 [api_impl.cc:2992] 619754 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.960.398 [awatchdog_core.c:175](tid:619754) create watchdog for id : 65543, tid : 619754, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.415 [api_impl.cc:3025] 619754 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.463 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.960.469 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.493 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.960.510 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 0.127 msec. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:44.960.509 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [WARNING] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.960.530 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:44.960.552 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.960.671 [device.cpp:143]619755 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.703 [api_impl.cc:2999] 619755 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.720 [device.cc:306] 619755 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.732 [runtime.cc:3393] 619755 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.748 [runtime.cc:3560] 619755 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.763 [api_impl.cc:2987] 619755 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.779 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.798 [api_impl.cc:2992] 619755 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:44.960.816 [awatchdog_core.c:175](tid:619755) create watchdog for id : 65543, tid : 619755, timeout : 300s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:44.960.831 [api_impl.cc:3025] 619755 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.960.843 [acl.cpp:115]619755 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.960.858 [acl.cpp:121]619755 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:44.960.870 [device.cpp:154]619755 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.124.183 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.124.183 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.124.207 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.124.216 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.220 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.234 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.239 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.261 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.124.267 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.124.275 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.124.282 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.124.290 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.297 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.124.306 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.292.730 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.292.773 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.292.794 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.292.807 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.292.833 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.292.915 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.308.361 [device.cpp:231]619755 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.407 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.573 [api_c_soc.cc:70] 619755 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.308.643 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.664 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.679 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.693 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.703 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.308.717 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=1. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.308.944 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=1.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.308.980 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.011 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.026 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.040 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.051 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.063 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=2. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.309.343 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=2.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.309.376 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.395 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.409 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.423 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.436 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.447 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=3. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.309.895 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=3.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.309.929 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.946 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.959 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.973 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.309.989 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.003 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=4. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.310.616 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=4.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.310.650 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.668 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.681 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.692 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.705 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.310.717 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=5. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.311.610 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=5.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.311.647 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.311.663 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.311.675 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.311.687 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.311.701 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.311.712 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=6. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.312.668 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=6.) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.312.704 [memory.cpp:559]619755 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.312.721 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.312.735 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.312.747 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.312.760 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.312.774 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=7. [INFO] DRV(619419,python3.9):2025-04-03-09:35:45.313.806 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=7.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.313.848 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.313.961 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.017 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.072 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.121 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.140 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.162 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.181 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.202 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.242 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.266 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.282 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.312 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.348 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.366 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.382 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.402 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.440 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.471 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.487 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.506 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.520 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.538 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.554 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.580 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.623 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.652 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.672 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.689 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.706 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.725 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.743 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.762 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.778 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.807 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.827 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.844 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.859 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.886 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.905 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.942 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.967 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.314.985 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.003 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.019 [runtime.cc:1664] 619755 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.036 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.062 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.080 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.100 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.138 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.163 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.181 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.194 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.209 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.231 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.254 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.268 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.282 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.300 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.317 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.349 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.362 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.375 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.390 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.408 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.436 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.449 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.470 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.485 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.499 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.525 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.539 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.552 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.566 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.579 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.615 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.632 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.646 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.660 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.674 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.698 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.712 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.315.726 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.527 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.584 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.635 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.649 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.663 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.677 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.699 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.741 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.753 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.769 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.785 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.820 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.855 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.869 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.883 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.900 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.922 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.963 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.981 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.316.998 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.013 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.031 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.070 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.086 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.099 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.116 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.142 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.183 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.195 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.208 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.223 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.239 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.276 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.290 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.303 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.319 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.345 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.388 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.403 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.416 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.430 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.447 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.485 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.497 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.510 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.526 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.548 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.592 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.606 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.621 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.638 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.654 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.692 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.706 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.719 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.733 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.754 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.794 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.809 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.824 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.838 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.854 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.895 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.911 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.941 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.955 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.317.976 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.014 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.030 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.043 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.058 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.074 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.113 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.128 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.141 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.155 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.178 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.217 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.232 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.245 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.259 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.277 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.317 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.333 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.345 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.360 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.379 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.419 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.436 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.450 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.473 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.491 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.528 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.544 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.557 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.572 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.588 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.631 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.646 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.659 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.674 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.693 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.733 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.748 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.761 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.775 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.793 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.831 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.846 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.858 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.318.913 [api_impl.cc:1990] 619755 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.319.225 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.319.257 [api_impl.cc:2664] 619755 MemSetSync: destMax=423624704, value=0, count=423624704 [2025-04-03 09:35:44.940183] [debug] [619419] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.940198] [debug] [619419] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:44.960990] [warn] [619755] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:44.961026] [debug] [619755] [lcal_sock_exchange.cpp:140] curRank: 0 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:44.961052] [info] [619755] [lcal_sock_exchange.cpp:198] The server is listening! ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.59032] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 5 [2025-04-03 09:35:45.93830] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 3 [2025-04-03 09:35:45.151903] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 2 [2025-04-03 09:35:45.223286] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 1 [2025-04-03 09:35:45.257091] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 7 [2025-04-03 09:35:45.258352] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 6 [2025-04-03 09:35:45.308028] [debug] [619755] [lcal_sock_exchange.cpp:249] Server side recv rank id 4 [2025-04-03 09:35:45.308127] [debug] [619755] [lcal_sock_exchange.cpp:93] rank:0 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308329] [debug] [619755] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 0 rank :0 rankSize :8 [2025-04-03 09:35:45.308520] [debug] [619755] [lcal_comm.cpp:406] rank 0 devId: 0, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308530] [info] [619755] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308539] [info] [619755] [lcal_comm.cpp:242] rank 0/8 running devId:0 [2025-04-03 09:35:45.308593] [debug] [619755] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308632] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308970] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 2, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309366] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309918] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310640] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311639] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312693] [debug] [619755] [lcal_comm.cpp:340] 0 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313824] [debug] [619755] [lcal_comm.cpp:372] EnablePeerAccess succeed0 [2025-04-03 09:35:45.318874] [debug] [619755] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 0 [2025-04-03 09:35:45.318899] [debug] [619755] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.319245] [debug] [619755] [lcal_comm.cpp:455] peerMem[rank0], allocate finished. [2025-04-03 09:35:45.340282] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340310] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340321] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340329] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340337] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340346] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340354] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340361] [debug] [619755] [lcal_comm.cpp:472] rank : 0, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340382] [debug[INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.446 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619423, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.469 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619427, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.486 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619431, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.511 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619435, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.529 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619439, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.544 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619443, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.564 [npu_driver.cc:3571] 619755 SetIpcMemPid: set ipc mem pid success, name=0009739b000000000000031b0000ffc00000, pid[0]=619447, num=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.340.905 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.341.748 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.341.768 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.342.317 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973a300000000000003180200ffc80002. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.342.333 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.342.804 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.342.821 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.357.925 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.357.942 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.359.009 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.359.028 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.359.801 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.359.817 [api_impl.cc:4286] 619755 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.305 [npu_driver.cc:3195] 619755 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.344 [api_impl.cc:1990] 619755 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.371 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.360.388 [memory.cpp:421]619755 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:45.360.665 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.360.701 [device.cpp:143]619755 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.722 [api_impl.cc:2999] 619755 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.745 [device.cc:306] 619755 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.758 [runtime.cc:3393] 619755 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.772 [runtime.cc:3560] 619755 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.784 [api_impl.cc:2987] 619755 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.796 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.813 [api_impl.cc:2992] 619755 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.827 [api_impl.cc:3025] 619755 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.360.840 [acl.cpp:115]619755 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.360.854 [acl.cpp:121]619755 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.360.867 [device.cpp:154]619755 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [WARNING] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:45.360.898 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.360.918 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.360.938 [adapter_rts.cc:198] [619755][hrtGetDeviceRefresh]deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.360.954 [op_base.cc:1091] [619755]Entry-HcclCommInitRootInfoInner:ranks[8], rank[0], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.360.972 [op_base.cc:128] [619755][HcclGetDeviceId] deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.032 [rank_consistentcy_checker.cc:32] [619755][GetInstance] get deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.361.075 [api_c_soc.cc:70] 619755 rtGetSocVersion: soc version is Ascend910B3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.361.092 [api_impl.cc:3100] 619755 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.361.106 [api_impl.cc:3111] 619755 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.361.122 [api_impl.cc:3130] 619755 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.161 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.177 [network_manager.cc:157] [619755]NetworkManager: init nic, nicPosition[0] ref[2], skip. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.192 [ra_host.c:1764]tid:619755,ra_get_ifnum(1764) : Input parameters: phy_id[0], nic_position:[0], is_all:[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.208 [ra_peer.c:859]tid:619755,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[0] start [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.224 [ra_peer.c:867]tid:619755,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[0] succ [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.240 [adapter_hccp.cc:1378] [619755][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.278 [ra_host.c:1814]tid:619755,ra_get_ifaddrs(1814) : Input parameters: phy_id[0], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.294 [ra_peer.c:874]tid:619755,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[0] start [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.307 [rs_socket.c:2743]tid:619755,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.320 [rs_socket.c:2743]tid:619755,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.330 [rs_socket.c:2743]tid:619755,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.343 [rs_socket.c:2749]tid:619755,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.353 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.365 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.379 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.392 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.403 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.415 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.426 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.439 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.449 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.460 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.469 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.481 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.494 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.507 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.517 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.530 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.543 [rs_socket.c:2749]tid:619755,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.554 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.565 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.578 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.589 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.602 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.614 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.636 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.647 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.659 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.670 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.681 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.692 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.704 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.715 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.727 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.739 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.752 [rs_socket.c:2749]tid:619755,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.765 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.778 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.790 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.799 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.812 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.824 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.834 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.847 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.857 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.866 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.877 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.889 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.902 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.914 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.926 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.937 [rs_socket.c:2751]tid:619755,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:45.361.954 [ra_peer.c:882]tid:619755,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[0] succ [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.972 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.361.986 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.009 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.023 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.037 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.052 [adapter_hccp.cc:1407] [619755][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.072 [sal.cc:365] [619755]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.083 [sal.cc:432] [619755]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.101 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.113 [network_manager.cc:1257] [619755]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe9808cb40] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.161 [adapter_hal.cc:298] [619755]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.185 [sal.cc:593] [619755]IsHostUseDevNic[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.200 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.215 [network_manager.cc:169] [619755]NetworkManager: devicePhyId[4294967295], devicePhyId_[0] deviceLogicId_[0], hasBackup[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.292 [tsd_client.cpp:189][TsdCapabilityGet][tid:619755] TsdCapabilityGet Begin. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.310 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.328 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.342 [process_mode_manager.cpp:1042][CapabilityGet][tid:619755] [ProcessModeManager] enter into CapabilityGet process deviceId[0] type[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.359 [adapter_tdt.cc:66] [619755]Get TsdCapability success. deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.372 [network_manager.cc:175] [619755][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:45.362.391 [network_manager.cc:99] [619755][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.407 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.420 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.432 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619755] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.445 [process_mode_manager.cpp:386][InitTsdClient][tid:619755] [TsdClient] tsd client has been inited before [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.465 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619755] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619419,python3.9):2025-04-03-09:35:45.362.483 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619755] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.497 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619755] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.522 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619755] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.536 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619755] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.560 [version_verify.cpp:114][SpecialFeatureCheck][tid:619755] VersionVerify: previous type[48], supported [INFO] TDT(619419,python3.9):2025-04-03-09:35:45.362.626 [hdc_client.cpp:154][TsdRecvData][tid:619755] recv timeout is:150000 ms [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.636.174 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.636.174 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.636.200 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.636.209 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.215 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.224 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.229 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.238 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.636.244 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.636.251 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.636.259 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.636.263 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.279 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.636.280 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.168 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.168 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.170 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.171 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.171 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:45.764.192 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.290 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.308 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.322 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.326 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.337 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.344 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.350 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.357 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.363 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.370 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.376 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.385 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.390 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.400 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.416 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.423 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.434 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.440 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.447 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.454 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.464 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.482 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.488 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.495 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.502 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.506 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.513 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.522 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.525 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.532 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.540 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.552 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:45.764.557 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.567 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.573 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:45.764.587 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.148.169 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.148.169 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.148.194 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.148.200 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.206 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.216 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.223 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.231 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.148.235 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.148.245 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.148.259 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.148.266 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.276 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.148.279 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.508.730 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:46.508.758 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:46.508.773 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:46.508.792 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.165 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.178 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.178 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.178 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.182 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.596.185 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.266 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.284 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.289 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.301 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.309 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.313 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.326 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.330 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.335 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.339 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.361 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.365 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.368 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.372 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.383 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.387 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.392 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.399 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.404 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.403 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.416 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.421 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.429 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.433 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.434 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.436 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.446 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.447 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.454 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.464 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.474 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.479 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.596.487 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.493 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.499 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.596.530 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.660.176 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.660.176 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.660.208 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:46.660.217 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.232 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.236 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.248 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.257 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.660.264 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.660.271 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.660.279 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:46.660.285 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.293 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:46.660.299 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.172.178 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.172.178 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.172.203 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.172.208 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.218 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.225 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.232 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.239 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.172.245 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.172.251 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.172.273 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.172.275 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.288 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.172.291 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.177 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.178 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.179 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.179 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.180 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.428.194 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.263 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.276 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.290 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.294 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.305 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.310 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.319 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.323 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.328 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.330 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.333 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.343 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.346 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.359 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.377 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.380 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.388 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.391 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.395 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.406 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.414 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.418 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.421 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.423 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.430 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.436 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.443 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.450 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.453 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.468 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.474 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.479 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.428.484 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.496 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.502 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.428.513 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.684.169 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.684.172 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.684.192 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:47.684.211 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.215 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.229 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.235 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.242 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.684.249 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.684.257 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.684.262 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:47.684.271 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.277 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.684.289 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.722.528 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.722.535 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619755] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.722.561 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619755] [TsdClient] DeviceMsgProc recvMsg realDeviceId[0] msgType[49] localDevId[0] rspCode[0]heterogeneousSubPid[346], tsdSupportLevel_[63] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.722.582 [hdc_client.cpp:172][TsdRecvData][tid:619755] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.722.597 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619755] OpenSubProc success stype:0, pid:346 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.722.611 [adapter_tdt.cc:39] [619755]Open TsdProcess success. deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.722.620 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.722.626 [network_manager.cc:118] [619755][TsdProcessOpen]hrtTsdProcessOpen success, subPid[346], devicePhyId_[0], deviceLogicId_[0], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.722.642 [network_manager.cc:180] [619755][NetworkManager][Init]open tsd by process success, devicePhyId[0], deviceLogicId_[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.722.690 [ra_host.c:205]tid:619755,ra_init(205) : Input parameters: phy_id[0], nic_position:[1] hdc_type:[18] [INFO] ROCE(619419,python3.9):2025-04-03-09:35:47.722.706 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.722.723 [devdrv_hccl.c:103][ascend][curpid:619419,619755][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619419) [INFO] ROCE(619419,python3.9):2025-04-03-09:35:47.722.748 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.722.767 [ra_hdc.c:580]tid:619755,ra_hdc_init(580) : hdc init start! logic id is 0, phy id is 0, hdc_type is 18 [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.722.942 [hdc_core.c:1982][ascend][curpid:619419,619755][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4262; pid=619419) [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.723.186 [ra_hdc.c:616]tid:619755,ra_hdc_init(616) : hdc init OK! phy_id[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.726.634 [adapter_hccp.cc:583] [619755]init ra success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.726.650 [network_manager.cc:222] [619755]NetworkManager nicDeploy[1] deviceLogicId_[0] devicePhyId_[0] init ra OK, nicSocketMap size[1], isHostUseDevNic[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.726.678 [topoinfo_detect.cc:534] [619755]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.726.699 [ra_host.c:1764]tid:619755,ra_get_ifnum(1764) : Input parameters: phy_id[0], nic_position:[1], is_all:[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.080 [adapter_hccp.cc:1538] [619755][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.096 [ra_host.c:1814]tid:619755,ra_get_ifaddrs(1814) : Input parameters: phy_id[0], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.448 [adapter_hccp.cc:1568] [619755][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth0] ip[192.168.100.101%eth0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.469 [adapter_hccp.cc:1568] [619755][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth0] ip[fe80::2287:ecff:fe02:45f2%eth0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.486 [topoinfo_detect.cc:577] [619755]select AF_INET family as device socket family. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.504 [topoinfo_detect.cc:756] [619755]no backup device ip: use 0 as device ip. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.518 [topoinfo_detect.cc:294] [619755][HCCL_TRACE]SetupAgent rankNum[8], rank[0], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[0], phydevId[0], deviceIp[192.168.100.101] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.551 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.568 [hccl_socket.cc:516] [619755][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.585 [hccl_socket.cc:217] [619755][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281468937489216], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.727.601 [adapter_hccp.cc:1050] [619755]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.618 [ra_host.c:759]tid:619755,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.642 [rs_socket.c:1810]tid:619755,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.661 [rs_socket.c:1819]tid:619755,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619419,python3.9):2025-04-03-09:35:47.727.680 [rs_socket.c:132]tid:619755,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.706 [rs_socket.c:1880]tid:619755,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.917 [rs_socket.c:1577]tid:619753,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:35807, fd:55 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.727.951 [rs_drv_socket.c:59]tid:619753,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:55 begin [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.051 [rs_drv_socket.c:98]tid:619753,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:35807, server addr 192.168.0.44:60008, fd:55 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.074 [rs_socket.c:1427]tid:619753,rs_conn_cost_time(1427) : socket [55] connect success! cost [0.368000] ms [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.117 [rs_socket.c:1392]tid:619753,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.151 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:56 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.210 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.003000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.236 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 56 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.254 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.728.271 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.728.794 [topoinfo_exchange_agent.cc:212] [619755]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.728.864 [topoinfo_exchange_agent.cc:236] [619755]local rank[0] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.728.883 [topoinfo_exchange_agent.cc:43] [619755]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.728.898 [topoinfo_exchange_agent.cc:345] [619755]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.012 [topoinfo_exchange_agent.cc:77] [619755]topo exchange client send rank basic info success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.037 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.070 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000000 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.094 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.110 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.729.123 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.736.877 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:57 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.736.962 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.059000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.016 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 57 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.037 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.051 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.066 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.104 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:58 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.131 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.005000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.150 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 58 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.163 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.174 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.186 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.198 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.217 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:59 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.238 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.004000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.254 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 59 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.269 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.294 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.306 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.317 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:59, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.331 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.350 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:60 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.379 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.004000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.395 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 60 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.410 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.424 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.437 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.448 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:59, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.460 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:60, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.473 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.493 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:61 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.515 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.004000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.533 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 61 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.546 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.566 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.578 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.590 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:59, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.593 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.603 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:60, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.622 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000001 / 8] success. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.622 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:61, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.644 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.737.650 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.662 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.682 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.701 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.726 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000003 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.751 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.767 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.781 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.799 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.858 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000004 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.878 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.894 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.906 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.936 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.958 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000006 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.976 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.737.991 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.738.003 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.738.021 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.119 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:62 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.167 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.027000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.184 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 62 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.197 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.209 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.224 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.236 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:59, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.248 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:60, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.262 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:61, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.274 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:62, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.738.289 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.101 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000007 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.122 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.145 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.156 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.175 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.195 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000002 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.211 [hccl_socket.cc:342] [619754][Accept]localIp[192.168.0.44%enp67s0f5], remoteIp[0.0.0.0], socket_handle[0xfffe9808cb40], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.226 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.739.238 [hccl_socket.cc:516] [619754][Get][NicHandleInfo]phyId[0], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.742.907 [rs_socket.c:829]tid:619752,rs_epoll_event_listen_in_handle(829) : [server]server_ip:192.168.0.44 server_port:60008 accept ok @ listen_fd:53, new fd:63 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.742.977 [rs_socket.c:694]tid:619752,rs_tcp_recv_tag_in_handle(694) : recv tag success, server:{192.168.0.44:60008} client:192.168.0.44 time_cost:0.045000ms tag_sync_time:1 tag_eintr_time:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.012 [rs_epoll.c:120]tid:619752,rs_wlist_check_conn_add(120) : [Server]chip_id 0, fd 63 accept, server ip:192.168.0.44, client ip:192.168.0.44, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.030 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:56, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.047 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:57, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.060 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:58, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.073 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:59, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.089 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:60, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.102 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:61, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.117 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:62, state:9, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.128 [rs_socket.c:62]tid:619752,show_conn_node(62) : current server ip: 192.168.0.44, client ip:192.168.0.44, fd:63, state:7, tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.743.154 [rs_epoll.c:329]tid:619752,rs_epoll_event_in_handle(329) : the fd:53 is tcp listened, no need to go on, ret:0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.743.478 [topoinfo_exchange_server.cc:130] [619754]listenSocket_->Accept completed. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.555 [topoinfo_exchange_server.cc:226] [619754]get remote rank[0000000000000005 / 8] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.571 [topoinfo_exchange_server.cc:53] [619754]cluster topo exchange server connect with all agent success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.595 [api_impl.cc:2096] 619754 HostMalloc: size=463 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.624 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=463 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.739 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.759 [topoinfo_exchange_server.cc:467] [619754]get rank basicInfo nicDeploy[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.776 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[1] deviceNum[1] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.795 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000000] rankId[0] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.812 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.829 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.897 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.916 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[2] deviceNum[2] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.744.934 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000001] rankId[1] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.948 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.744.967 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.029 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.049 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[3] deviceNum[3] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.066 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000002] rankId[2] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.080 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.098 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.161 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.178 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[4] deviceNum[4] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.195 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000003] rankId[3] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.209 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.227 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.300 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.320 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[5] deviceNum[5] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.337 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000004] rankId[4] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.351 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.369 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.429 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.446 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[6] deviceNum[6] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.462 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000005] rankId[5] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.476 [api_impl.cc:2096] 619754 HostMalloc: size=461 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.493 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=461 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.552 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.570 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[7] deviceNum[7] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.587 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000006] rankId[6] success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.601 [api_impl.cc:2096] 619754 HostMalloc: size=460 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.619 [api_error.cc:1260] 619754 HostMalloc: Host memory malloc succeed,size=460 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.745.680 [api_error.cc:1278] 619754 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.697 [topoinfo_exchange_server.cc:479] [619754]get rank basicInfo serverNum[1] rankNum[8] deviceNum[8] superPodNum[0], nicDeploy[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.714 [topoinfo_exchange_server.cc:372] [619754]GetRankBasicInfo from agentId[0000000000000007] rankId[7] success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.734 [topoinfo_exchange_server.cc:59] [619754]cluster topo exchange server get rank basic info from all agent success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.745.752 [topoinfo_exchange_dispatcher.cc:43] [619754][TopoInfoExchangeDispather][InitWorkerThread]calculate threadNum[1], rankNum[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.577 [topoinfo_exchange_dispatcher.cc:230] [619754][TopoInfoExchangeDispather][ProcessSend]sendOnce success, start epoll_wait. sendDoneCount[8], rankNum[8]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.800.606 [api_impl.cc:2096] 619755 HostMalloc: size=2485 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.613 [topoinfo_exchange_dispatcher.cc:35] [619754]cluster topo exchange worker broadcast topoinfo success, rankNum[8], threadNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.630 [topoinfo_exchange_server.cc:65] [619754]cluster topo exchange server send rank basic info to all agent success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.800.639 [api_error.cc:1260] 619755 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:47.800.647 [hccl_socket.cc:232] [619754][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.671 [network_manager.cc:69] [619754]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.800.690 [ra_host.c:911]tid:619754,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.800.708 [rs_socket.c:1262]tid:619754,rs_socket_listen_stop(1262) : listen [0] IP 0xffffffff2c00a8c0, ret_vnic -1 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.800.729 [rs_socket.c:746]tid:619754,rs_socket_listen_del_from_epoll(746) : IP:192.168.0.44 server_port:60008 listen_fd:53 del from epoll:51 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.800.761 [rs_socket.c:1292]tid:619754,rs_socket_listen_stop(1292) : IP(192.168.0.44) close listen fd:53 ! [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.780 [network_manager.cc:786] [619754]ip[192.168.0.44%enp67s0f5] port[60008] stop success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.797 [hccl_socket.cc:68] [619754][HcclSocket][DeInit] device[0] stops listen on ip[192.168.0.44%enp67s0f5], port[60008] success, socketType[2]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.817 [topoinfo_exchange_dispatcher.cc:145] [619754][TopoInfoExchangeDispather][PrepareEpollResource]wake all workers. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.800.932 [api_error.cc:1278] 619755 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.951 [topoinfo_exchange_agent.cc:80] [619755]topo exchange client get rank basic info success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.800.953 [topoinfo_exchange_server.cc:77] [619754]cluster topo exchange server completed, exit[0]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.800.976 [api_impl.cc:3419] 619754 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.800.992 [runtime.cc:3444] 619754 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.801.007 [api_impl.cc:3449] 619754 DeviceReset: Succ, userDeviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.010 [topoinfo_detect.cc:303] [619755]topo detect completed. myrank[0], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.059 [topoinfo_detect.cc:242] [619755]topoExchangeServer[60008] compeleted. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.078 [config.cc:198] [619755]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.101 [config.cc:210] [619755]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.114 [config.cc:210] [619755]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.129 [config.cc:210] [619755]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.142 [config.cc:210] [619755]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.154 [config.cc:210] [619755]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.166 [config.cc:210] [619755]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.178 [config.cc:210] [619755]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.198 [config.cc:210] [619755]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.801.212 [adapter_hccp.cc:1067] [619755]ra socket batch close fdhandle[0xfffe7c0549a0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.801.227 [ra_host.c:796]tid:619755,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.801.246 [rs_socket.c:1937]tid:619755,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:55, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.207 [adapter_hccp.cc:1099] [619755]ra socket batch close success,take time [46982]us [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:47.848.243 [hccl_socket.cc:232] [619755][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.262 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.279 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.292 [network_manager.cc:376] [619755]NetworkManager: deinit nic success, nicPosition[0] ref[1], skip. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.306 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.848.321 [ra_host.c:248]tid:619755,ra_deinit(248) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.848.338 [ra_hdc.c:651]tid:619755,ra_hdc_deinit(651) : hdc deinit start! phy_id[0] [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.848.472 [hdc_client.c:417][ascend][curpid:619419,619755][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4262) [INFO] ROCE(619419,python3.9):2025-04-03-09:35:47.848.534 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:47.848.549 [ra_hdc.c:685]tid:619755,ra_hdc_deinit(685) : hdc deinit OK! phy_id[0] [INFO] ROCE(619419,python3.9):2025-04-03-09:35:47.848.560 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.596 [tsd_client.cpp:189][TsdCapabilityGet][tid:619755] TsdCapabilityGet Begin. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.613 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.625 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.639 [process_mode_manager.cpp:1042][CapabilityGet][tid:619755] [ProcessModeManager] enter into CapabilityGet process deviceId[0] type[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.848.652 [adapter_tdt.cc:66] [619755]Get TsdCapability success. deviceLogicId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.666 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.677 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.691 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619755] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.708 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619755] add close subproc:346, proctype:0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.724 [version_verify.cpp:114][SpecialFeatureCheck][tid:619755] VersionVerify: previous type[60], supported [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.848.765 [hdc_client.cpp:154][TsdRecvData][tid:619755] recv timeout is:150000 ms [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:47.951.911 [awatchdog_monitor.c:160](tid:619459) thread 619754 not exist [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.981.439 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619755] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.981.465 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619755] [TsdClient] DeviceMsgProc recvMsg realDeviceId[0] msgType[61] localDevId[0] rspCode[0]heterogeneousSubPid[346], tsdSupportLevel_[63] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.981.483 [hdc_client.cpp:172][TsdRecvData][tid:619755] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.981.498 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619755] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.981.511 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619755] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.523 [adapter_tdt.cc:52] [619755]Open TsdProcess success. deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.538 [network_manager.cc:405] [619755][DeInit]finish CloseHccpProcess, phy_id[0], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.550 [network_manager.cc:409] [619755]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.562 [topoinfo_detect.cc:555] [619755]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.576 [topoinfo_detect.cc:321] [619755]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.588 [topoinfo_detect.cc:219] [619755]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.603 [rank_consistentcy_checker.cc:32] [619755][GetInstance] get deviceLogicId[0] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:47.981.617 [rank_consistentcy_checker.cc:144] [619755][Record][CannVersion] version information str is empty. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.629 [op_base.cc:2231] [619755]rank table is null, rankTableCrc is 0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.669 [op_base.cc:2219] [619755]params.id.internal [hccl-619419-0-1743644147-hccl_world_group] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.684 [op_base.cc:993] [619755]rootInfo[hccl-619419-0-1743644147-hccl_world_group], params.logiceDevice[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.695 [hccl_comm.cc:98] [619755]hcclComm init workmode [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.981.708 [hccl_comm.cc:954] [619755]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.982.566 [hccl_comm.cc:1365] [619755]RegistTaskAbortHandler begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.982.585 [task_abort_handler.cc:126] [619755]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.982.601 [api_impl.cc:4192] 619755 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.982.620 [runtime.cc:4698] 619755 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.982.661 [adapter_rts.cc:42] [619755]open so: libruntime.so [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.982.930 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.982.966 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.982.993 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.010 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.025 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.042 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.057 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.074 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.088 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.103 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.118 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.132 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.147 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.162 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.177 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.193 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.208 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.222 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.237 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.251 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.266 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.280 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.293 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.310 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.325 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.339 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.359 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.374 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.389 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.405 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.420 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.434 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.449 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.463 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.478 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.492 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.507 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.522 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.536 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.551 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.566 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.581 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.595 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.609 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.624 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.639 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.653 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.668 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.683 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.698 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.713 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.736 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.751 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.766 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.983.781 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.796 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.855 [hccl_communicator_attrs.cc:407] [619755]module[0] contains [8]devices [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.872 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.886 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.901 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.912 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.925 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.939 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.950 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.963 [hccl_communicator_attrs.cc:411] [619755]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.976 [hccl_communicator_attrs.cc:425] [619755]different module contains different numbers of cards:[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.983.990 [hccl_communicator_attrs.cc:426] [619755]different module contains different type of cards:[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.049 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.073 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.093 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.114 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.130 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.148 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.182 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.202 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.224 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.242 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.257 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.280 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.295 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.312 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.327 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.342 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.358 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.379 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.395 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.418 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.434 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.451 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.466 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.481 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.497 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.513 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.528 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.544 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.557 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.575 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.590 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.608 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.623 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.649 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.664 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.683 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.698 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.715 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.732 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.753 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.769 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.785 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.800 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.815 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.830 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.853 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.868 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.891 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.907 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.924 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.939 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.955 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.984.968 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.984.990 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.005 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.021 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.036 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.052 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.073 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.091 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.106 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.129 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.144 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.161 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.176 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.192 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.207 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.223 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.238 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.254 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.269 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.285 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.300 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.321 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.337 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.359 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.374 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.394 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.410 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.427 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.443 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.462 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.477 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.506 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.520 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.538 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.553 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.576 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.591 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.607 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.622 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.644 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.659 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.676 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.691 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.710 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.726 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.744 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.759 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.775 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.790 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.813 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.827 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.849 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.864 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.881 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.895 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.919 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.935 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.951 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.985.966 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.985.981 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.003 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.022 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.036 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.051 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.065 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.080 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.094 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.108 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.123 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.137 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.152 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.166 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.181 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.195 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.210 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.224 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.237 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.253 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.267 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.282 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.301 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.316 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.331 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.345 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.359 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.374 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.388 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.403 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.417 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.432 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.446 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.461 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.475 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.489 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.504 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.519 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.533 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.548 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.562 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.576 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.589 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.604 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.619 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.634 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.648 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.673 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.686 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.704 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.718 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.733 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.747 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.764 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.778 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.792 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.986.807 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.821 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.868 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.891 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.910 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.928 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.946 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.967 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.982 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.986.999 [hccl_communicator_attrs.cc:156] [619755][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.029 [hccl_communicator_attrs.cc:1021] [619755][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.048 [hccl_communicator.cc:1440] [619755][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.061 [hccl_communicator.cc:1450] [619755][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.077 [externalinput.cc:1445] [619755]InitParsesProtocol start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.091 [externalinput.cc:1462] [619755]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:47.987.114 [externalinput.cc:1411] [619755]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.139 [externalinput.cc:1413] [619755]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.154 [externalinput.cc:1467] [619755]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.200 [config.cc:557] [619755][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.220 [hccl_communicator.cc:668] [619755][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.252 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.272 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.289 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=1. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.311 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=1.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.327 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.339 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.352 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.367 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=2. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.382 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=2.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.396 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.408 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.421 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.435 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=3. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.449 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=3.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.464 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.475 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.488 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.508 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=4. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.522 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=4.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.537 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.548 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.561 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.576 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=5. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.590 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=5.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.604 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.617 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.630 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.645 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=6. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.660 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=6.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.674 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.685 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.698 [api_impl.cc:3520] 619755 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:47.987.713 [api_impl.cc:3168] 619755 EnableP2P: Enable P2P devIdDes=0, phyIdSrc=7. [INFO] DRV(619419,python3.9):2025-04-03-09:35:47.987.727 [devdrv_pcie.c:888][ascend][curpid:619419,619755][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=0; peer_phy_devid=7.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.742 [p2p_mgmt.cc:74] [619755]enable p2p: local logic id:0, remote physic id:7. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.765 [adapter_hal.cc:298] [619755]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.787 [sal.cc:593] [619755]IsHostUseDevNic[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.835 [adapter_hal.cc:298] [619755]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.856 [sal.cc:593] [619755]IsHostUseDevNic[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.869 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.881 [network_manager.cc:169] [619755]NetworkManager: devicePhyId[4294967295], devicePhyId_[0] deviceLogicId_[0], hasBackup[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.987.896 [tsd_client.cpp:189][TsdCapabilityGet][tid:619755] TsdCapabilityGet Begin. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.987.910 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.987.922 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.987.933 [process_mode_manager.cpp:1042][CapabilityGet][tid:619755] [ProcessModeManager] enter into CapabilityGet process deviceId[0] type[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.953 [adapter_tdt.cc:66] [619755]Get TsdCapability success. deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.964 [network_manager.cc:175] [619755][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:47.987.982 [network_manager.cc:99] [619755][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.000 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.012 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619755] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.025 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619755] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.036 [process_mode_manager.cpp:386][InitTsdClient][tid:619755] [TsdClient] tsd client has been inited before [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.049 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619755] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619419,python3.9):2025-04-03-09:35:47.988.067 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619755] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.080 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619755] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.094 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619755] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.113 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619755] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619419,python3.9):2025-04-03-09:35:47.988.169 [hdc_client.cpp:154][TsdRecvData][tid:619755] recv timeout is:150000 ms [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.196.177 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.196.180 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.196.205 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.196.213 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.220 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.231 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.238 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.244 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.196.270 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.196.273 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.196.285 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.196.292 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.305 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.196.309 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.171 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.172 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.173 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.174 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.174 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.260.175 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.276 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.292 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.307 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.313 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.320 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.329 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.332 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.343 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.346 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.355 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.359 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.363 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.390 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.396 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.401 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.409 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.417 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.422 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.430 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.434 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.440 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.444 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.449 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.456 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.462 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.473 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.476 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.481 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.488 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.492 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.502 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.508 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.260.517 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.525 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.534 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.260.546 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.708.175 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.708.175 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.708.208 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:48.708.217 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.221 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.240 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.238 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.708.250 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.255 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.708.261 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.708.269 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.277 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:48.708.282 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.708.297 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:48.940.717 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:48.940.754 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:48.940.771 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:48.940.785 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.174 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.174 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.175 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.175 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.176 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.092.179 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.265 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.281 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.284 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.293 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.298 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.303 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.311 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.318 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.321 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.322 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.330 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.335 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.340 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.350 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.354 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.354 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.368 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.372 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.382 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.385 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.389 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.394 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.399 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.408 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.411 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.419 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.424 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.442 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.450 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.452 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.464 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.467 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.092.477 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.483 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.488 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.092.502 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.220.174 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.220.174 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.220.193 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.220.197 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.211 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.216 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.224 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.230 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.220.237 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.220.241 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.220.250 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.220.256 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.264 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.220.269 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.732.173 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.732.176 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.732.198 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.732.205 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.215 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.223 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.229 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.237 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.732.244 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.732.250 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.732.256 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.732.260 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.269 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.732.272 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.173 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.174 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.177 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.179 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.177 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:49.924.181 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.277 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.291 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.305 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.323 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.334 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.364 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.376 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.381 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.388 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.374 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.395 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.398 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.419 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.425 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.416 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.434 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.438 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.440 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.463 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.470 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.448 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.458 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.475 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.476 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.490 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.500 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.512 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.521 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.525 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.610 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.623 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.564 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.575 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.924.655 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.667 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.924.679 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] TDT(619419,python3.9):2025-04-03-09:35:49.955.450 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619755] [ServerToClientMsgProc] [sessionID=1] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.955.451 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:49.955.470 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619755] [TsdClient] DeviceMsgProc recvMsg realDeviceId[0] msgType[49] localDevId[0] rspCode[0]heterogeneousSubPid[366], tsdSupportLevel_[63] [INFO] TDT(619419,python3.9):2025-04-03-09:35:49.955.488 [hdc_client.cpp:172][TsdRecvData][tid:619755] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:49.955.504 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619755] OpenSubProc success stype:0, pid:366 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.955.518 [adapter_tdt.cc:39] [619755]Open TsdProcess success. deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.955.523 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.955.529 [network_manager.cc:118] [619755][TsdProcessOpen]hrtTsdProcessOpen success, subPid[366], devicePhyId_[0], deviceLogicId_[0], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.955.546 [network_manager.cc:180] [619755][NetworkManager][Init]open tsd by process success, devicePhyId[0], deviceLogicId_[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:49.955.594 [ra_host.c:205]tid:619755,ra_init(205) : Input parameters: phy_id[0], nic_position:[1] hdc_type:[18] [INFO] ROCE(619419,python3.9):2025-04-03-09:35:49.955.613 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619419,python3.9):2025-04-03-09:35:49.955.629 [devdrv_hccl.c:103][ascend][curpid:619419,619755][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619419) [INFO] ROCE(619419,python3.9):2025-04-03-09:35:49.955.645 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:49.955.658 [ra_hdc.c:580]tid:619755,ra_hdc_init(580) : hdc init start! logic id is 0, phy id is 0, hdc_type is 18 [INFO] DRV(619419,python3.9):2025-04-03-09:35:49.955.805 [hdc_core.c:1982][ascend][curpid:619419,619755][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4270; pid=619419) [INFO] HCCP(619419,python3.9):2025-04-03-09:35:49.956.181 [ra_hdc.c:616]tid:619755,ra_hdc_init(616) : hdc init OK! phy_id[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.597 [adapter_hccp.cc:583] [619755]init ra success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.614 [network_manager.cc:222] [619755]NetworkManager nicDeploy[1] deviceLogicId_[0] devicePhyId_[0] init ra OK, nicSocketMap size[1], isHostUseDevNic[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.645 [hccl_communicator_attrs.cc:1307] [619755][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.659 [hccl_communicator_attrs.cc:1275] [619755]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.678 [hccl_communicator.cc:2259] [619755][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.701 [adapter_hccp.cc:1650] [619755]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.898 [adapter_hccp.cc:2343] [619755]Get available Vnic info success, deviceIdType[0] deviceId[0x0] Vnic ip[192.168.2.199] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.944 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:49.959.965 [ra_host.c:280]tid:619755,ra_socket_init(280) : socket init:mode=1 phy_id=0 family=2 ip=192.168.2.199 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.982 [adapter_hccp.cc:806] [619755]socket init success, ip[3338840256] device id[0], socketHandle[0xfffe7c07c8c0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.959.996 [network_manager.cc:692] [619755]devicePhyId[0], ip[192.168.2.199] socket init OK [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.010 [network_manager.cc:432] [619755][Start][Vnic]trying to listen on ip[192.168.2.199] port[16666]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:49.960.025 [ra_host.c:875]tid:619755,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[0], local_ip[192.168.2.199], port[16666] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.338 [network_manager.cc:448] [619755][Start][Vnic]Listen on ip[192.168.2.199], port[16666] sucess, devPhyId[0], devLogicId[0], isAutoPort[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.357 [network_manager.cc:451] [619755]NetworkManager devicePhyId_[0] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.373 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.387 [hccl_socket.cc:106] [619755][HcclSocket][Listen] device[0] listens on ip[192.168.2.199] port[16666] success, socketType[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.403 [hccl_socket_manager.cc:56] [619755][Init][Server]ip[192.168.2.199] port[16666] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.421 [hccl_communicator.cc:1548] [619755][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.199] port[16666] success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.436 [hccl_communicator_attrs.cc:1307] [619755][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.448 [hccl_communicator_attrs.cc:1275] [619755]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.463 [hccl_communicator.cc:1565] [619755]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.478 [network_manager.cc:69] [619755]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.490 [hccl_communicator.cc:2028] [619755]profiling config information:options[], mode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.623 [adapter_trace.cc:21] [619755]Call TraceCreate, return value[0] [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:49.960.645 [atrace_api.c:61](tid:619755) AtraceCreateWithAttr start. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:49.960.730 [trace_rb_log.c:45](tid:619755) init ring buffer time finished, realTime 1743644149960723200, monotonicTime 4211917065642, freq 99988 kHz, minutesWest 480 minutes [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:49.960.756 [trace_rb_log.c:137](tid:619755) [HCCL_619755_0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:49.960.773 [tracer_mgr_operate.c:66](tid:619755) create object HCCL_619755_0 success, exitSave(true), noLock(false). [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:49.960.788 [atrace_api.c:63](tid:619755) AtraceCreateWithAttr end. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.798 [adapter_trace.cc:70] [619755]Call TraceCreateWithAttr, return value[0xaaaab0f64a48], Params: objName[HCCL_619755_0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.856 [rank_consistentcy_checker.cc:32] [619755][GetInstance] get deviceLogicId[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.960.893 [api_impl.cc:1486] 619755 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.960.919 [npu_driver.cc:4604] 619755 GetMaxStreamAndTask: deviceId=0, tsId=0, streamCount=1984. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.935 [task_exception_handler.cc:1212] [619755]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.960.955 [hccl_communicator_attrs.cc:1099] [619755][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.961.020 [api_impl.cc:4219] 619755 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.961.043 [command_handle.cc:25] [619755][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.961.058 [command_handle.cc:99] [619755][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.961.169 [profiler_manager_impl.cc:76] [619755][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.961.187 [hccl_communicator.cc:2193] [619755][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.961.199 [hccl_communicator.cc:803] [619755]Will use ffts mode. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.961.228 [api_impl.cc:3138] 619755 GetDeviceIndexByPhyId: get Index by PhyId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.961.244 [api_impl.cc:3149] 619755 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.961.260 [api_impl.cc:3157] 619755 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.961.272 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.962.760 [api_impl.cc:3476] 619755 GetDeviceInfo: get device info, deviceId=0 [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:49.962.806 [tbe_vector_reduce.cc:508] [619755]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.963.867 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.026 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.964.045 [dispatcher.cc:157] [619755][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.064 [api_impl.cc:3138] 619755 GetDeviceIndexByPhyId: get Index by PhyId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.080 [api_impl.cc:3149] 619755 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.091 [api_impl.cc:3157] 619755 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.103 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.137 [api_impl.cc:1990] 619755 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.964.987 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.965.046 [api_impl.cc:1990] 619755 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.965.070 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.965.102 [api_impl.cc:1990] 619755 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.965.121 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.151 [opexecounter.cc:71] [619755]alloc counter mem resource. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.189 [hccl_communicator_attrs.cc:1368] [619755][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.237 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.260 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.284 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.301 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.314 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.327 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.343 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.357 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.371 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.386 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.402 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.413 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.428 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.442 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.456 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.485 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.498 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.512 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.527 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.540 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.553 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.565 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.579 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.594 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.606 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.622 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.634 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.649 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.662 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.677 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.690 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.706 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.717 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.731 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.746 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.758 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.773 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.787 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.808 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.821 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.836 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.848 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.865 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.878 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.890 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.905 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.918 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.931 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.947 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.960 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.976 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.965.989 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.005 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.016 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.032 [alg_configurator.cc:68] [619755][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.045 [alg_configurator.cc:241] [619755][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.059 [alg_configurator.cc:134] [619755]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.082 [alg_configurator.cc:449] [619755][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.098 [alg_configurator.cc:42] [619755]alg configurator init success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.133 [topo_info_extractor.cc:66] [619755]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[0], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.172 [topo_info_extractor.cc:155] [619755][SetRankInfo]rankData_: userRank[0], devicePhyId[0], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.211 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.242 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.269 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.290 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.307 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.326 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.358 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.379 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.396 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.417 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.434 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.455 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.966.471 [api_error.cc:3478] 619755 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.489 [adapter_rts.cc:940] [619755][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.510 [topo_info_extractor.cc:348] [619755]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.647 [topo_info_extractor.cc:888] [619755]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[0], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.685 [topo_info_extractor.cc:585] [619755][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.710 [topo_info_extractor.cc:646] [619755][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.729 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/1; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.748 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.766 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.784 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.808 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.823 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.841 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.859 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.872 [topo_info_extractor.cc:689] [619755]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[0], userRankSize[8], plane size[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.885 [topo_info_extractor.cc:697] [619755][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.902 [topo_info_extractor.cc:734] [619755]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 0/0/0/; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.920 [topo_info_extractor.cc:739] [619755]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[0], userRankSize[8], plane size[1] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.936 [topo_info_extractor.cc:585] [619755][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.951 [topo_info_extractor.cc:646] [619755][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.967 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/1; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.966.982 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.000 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.018 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.034 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.051 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.069 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.087 [topo_info_extractor.cc:685] [619755]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.101 [topo_info_extractor.cc:689] [619755]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[0], userRankSize[8], plane size[8] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.153 [topo_info_extractor.cc:790] [619755]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId0/0; ] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.174 [topo_info_extractor.cc:519] [619755]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.967.326 [api_impl.cc:1990] 619755 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.967.354 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.406 [hccl_impl.cc:59] [619755]start hccl resources build:get deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.967.535 [hccl_impl.cc:177] [619755]hcclImpl init success. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:49.967.567 [hccl_aiv.cc:279] [619755][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.360 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.416 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.528 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.549 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.568 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.592 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.616 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.713 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.730 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.748 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.765 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.788 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.888 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.903 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.920 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.937 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.970.960 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.051 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.066 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.093 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.108 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.130 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.224 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.240 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.257 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.272 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.291 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.383 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.398 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.412 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.427 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.448 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.542 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.558 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.576 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.591 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.613 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.702 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.717 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.731 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.748 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.767 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.860 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.874 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.891 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.908 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.971.936 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.025 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.040 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.058 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.074 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.096 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.196 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.213 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.229 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.245 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.266 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.359 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.374 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.391 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.405 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.427 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.520 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.536 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.551 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.569 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.587 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.678 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.694 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.708 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.724 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.743 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.841 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.856 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.870 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.888 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.907 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.972.999 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.014 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.029 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.046 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.065 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.156 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.171 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.183 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.198 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.219 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.310 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.325 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.344 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.360 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.382 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.474 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.490 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.502 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.518 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.539 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.631 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.646 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.670 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.686 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.708 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.802 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.817 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.831 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.846 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.867 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.955 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.971 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.973.984 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.001 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.020 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.114 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.129 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.142 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.159 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.182 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.275 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.290 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.306 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.324 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.345 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.437 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.452 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.467 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.491 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.510 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.602 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.617 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.635 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.651 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.670 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.764 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.780 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.794 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.811 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.830 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.919 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.934 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.948 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.965 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.974.984 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.078 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.093 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.110 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.124 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.145 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.235 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.250 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.263 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.280 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.308 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.402 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.417 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.434 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.451 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.470 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.561 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.576 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.594 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.608 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.629 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.717 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.732 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.749 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.763 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.784 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.873 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.889 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.906 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.921 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.975.942 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.035 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.051 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.069 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.084 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.105 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.211 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.234 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.251 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.269 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.290 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.382 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.397 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.411 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.428 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.447 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.542 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.559 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.572 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.590 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.611 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.704 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.719 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.737 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.753 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.774 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.865 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.881 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.898 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.912 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.976.934 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.023 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.039 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.062 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.077 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.099 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.191 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.207 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.223 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.239 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.257 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.347 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.362 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.376 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.393 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.416 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.505 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.519 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.532 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.550 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.570 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.662 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.677 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.690 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.707 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.729 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.822 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.837 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.855 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.879 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.902 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.977.991 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.006 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.021 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.037 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.057 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.147 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.162 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.175 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.193 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.212 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.300 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.316 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.330 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.347 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.366 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.458 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.473 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.490 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.505 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.526 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.616 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.631 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.644 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.661 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.680 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.775 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.791 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.805 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.821 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.840 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.936 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.951 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.964 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.978.981 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.000 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.097 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.112 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.126 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.142 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.164 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.252 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.268 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.282 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.300 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.321 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.410 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.426 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.439 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.457 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.478 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.568 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.591 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.609 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.624 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.645 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.735 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.751 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.766 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.782 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.803 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.892 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.908 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.925 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.939 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.979.961 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.051 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.067 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.084 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.099 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.120 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.228 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.245 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.261 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.276 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.298 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.387 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.403 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.417 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.440 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.460 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.550 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.567 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.584 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.600 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.620 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.709 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.725 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.742 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.758 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.778 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.871 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.887 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.902 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.919 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.980.940 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.029 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.046 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.063 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.079 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.100 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.189 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.205 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.222 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.237 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.265 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.355 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.371 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.388 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.403 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.424 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.516 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.531 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.545 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.562 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.583 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.677 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.693 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.707 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.724 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.746 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.836 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.852 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.869 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.885 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.906 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.981.995 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.012 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.028 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.043 [runtime.cc:1673] 619755 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.064 [elf.cc:566] 619755 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.157 [runtime.cc:1688] 619755 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.177 [runtime.cc:1953] 619755 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.191 [runtime.cc:1897] 619755 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619419,python3.9):2025-04-03-09:35:49.982.214 [devmm_svm.c:1972][ascend][curpid:619419,619755][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.982.231 [hdc.cc:83] [619755][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.245 [api_impl.cc:1990] 619755 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.270 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.760 [api_impl.cc:1990] 619755 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.781 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.802 [api_impl.cc:2664] 619755 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619419,python3.9):2025-04-03-09:35:49.982.964 [devmm_svm.c:1972][ascend][curpid:619419,619755][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.982.983 [hdc.cc:83] [619755][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.982.994 [api_impl.cc:1990] 619755 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.010 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.033 [api_impl.cc:2096] 619755 HostMalloc: size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.051 [api_error.cc:1260] 619755 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.200 [api_impl.cc:1990] 619755 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.271 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.292 [api_impl.cc:2664] 619755 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.431 [api_impl.cc:1990] 619755 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.451 [api_error.cc:1218] 619755 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.470 [api_impl.cc:2664] 619755 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.496 [peterson_lock.cc:69] [619755][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.512 [peterson_lock.cc:47] [619755][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.541 [hccl_comm.cc:129] [619755]hcclCommInitInfo:commId[hccl-619419-0-1743644147-hccl_world_group], rank[0], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[0], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.564 [hccl_comm.cc:175] [619755][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.589 [topo_matcher.cc:528] [619755][SetAivMode]AivMode is set to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.602 [topo_matcher.cc:540] [619755][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.624 [hcom_plugin.cc:349] [619755][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.983.663 [runtime.cc:5239] 619755 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.678 [adapter_rts.cc:198] [619755][hrtGetDeviceRefresh]deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.697 [hcom_common.cc:85] [619755][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.717 [op_base.cc:128] [619755][HcclGetDeviceId] deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.734 [hccl_comm.cc:1296] [619755]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffe92fdccb8] topoSize[2] [INFO] GE(619419,python3.9):2025-04-03-09:35:49.983.760 [hcom_topo_info.cc:24]619755 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.777 [hcom_plugin.cc:371] [619755][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.798 [topoinfo_detect.cc:555] [619755]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.811 [topoinfo_detect.cc:321] [619755]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.823 [topoinfo_detect.cc:219] [619755]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.845 [op_base.cc:1050] [619755][InitCommRootInfo]Init success, rankNum[8], rank[0], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.864 [profiler_base.cc:342] [619755]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.881 [op_base.cc:1107] [619755][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4622966]us, rankNum[8], rank[0] [WARNING] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:49.983.947 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619419,python3.9):2025-04-03-09:35:49.983.974 [op_base.cc:853] [619755]HcclGetCommName input handle=0xfffe7c022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:49.984.005 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 0 [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:49.984.028 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 0, group name: hccl_world_group [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:49.984.125 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.984.144 [device.cpp:183]619755 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.984.192 [api_impl.cc:3419] 619755 DeviceReset: reset device, device_id=0. [INFO] DEVICE(619419,fffe927cefa0,python3.9):2025-04-03-09:35:49.984.193 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 0, group name: hccl_world_group [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.984.211 [runtime.cc:3444] 619755 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.984.229 [api_impl.cc:3449] 619755 DeviceReset: Succ, userDeviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.984.240 [device.cpp:189]619755 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 0. [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:49.984.266 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:49.984.332 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5023.73 msec. [WARNING] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:49.984.362 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.984.426 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5030.29 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.984.468 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5030.35 msec. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.984.487 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.984.512 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9138.67 msec. [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.984.548 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.327 [device.cpp:143]619419 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.362 [api_impl.cc:2999] 619419 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.382 [device.cc:306] 619419 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.397 [runtime.cc:3393] 619419 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.415 [runtime.cc:3560] 619419 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.429 [api_impl.cc:2987] 619419 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.444 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.459 [api_impl.cc:2992] 619419 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.488 [api_impl.cc:3025] 619419 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.505 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.520 [acl.cpp:121]619419 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.534 [device.cpp:154]619419 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:49.999.573 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.594 [context.cpp:174]619419 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:49.999.616 [api_impl.cc:6913] 619419 CtxSetSysParamOpt: curCtx = 0xaaaac1adc2f0, configOpt=0, configVal=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:49.999.632 [context.cpp:158]619419 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619419,python3.9):2025-04-03-09:35:50.000.228 [op_base.cc:1214] [619419][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619419,python3.9):2025-04-03-09:35:50.000.252 [op_base.cc:128] [619419][HcclGetDeviceId] deviceLogicId[0] [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.000.278 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.000.297 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.000.316 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.000.357 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.000.898 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.404 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.480 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.680 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 208.82 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.719 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.763 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 42.51 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.788 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.809 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 21.13 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.828 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.908 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 76.92 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.001.958 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.555 msec. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.068 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.131 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 9.081 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.210 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 52.86 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.256 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.184 msec. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.285 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.325 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.377 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 49.24 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.398 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 31.67 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.455 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.475 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 19.35 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.493 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.530 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 34.48 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.573 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.284 msec. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.607 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.726 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.805 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 77.27 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.868 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 36.9 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.933 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 30.08 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.002.994 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 36.91 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 35.16 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.192 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 112.46 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.221 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 2.86 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.243 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 50.77 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 44.51 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.408 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.473 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 43.71 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.845 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 343.61 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.868 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.934 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 44.11 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.003.988 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.377 msec. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.008 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.064 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 1.994 msec. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.490 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 52.67 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.519 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.609 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 87.21 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.642 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.697 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 50.26 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.719 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.753 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 31.94 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.790 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.811 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 36.45 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.840 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 8.34 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.888 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 47.86 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.929 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 18.93 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.968 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.004.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 18.38 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.019 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.039 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 31.83 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.058 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.077 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 18.77 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.096 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.114 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 17.77 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.142 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.187 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 42 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.235 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.255 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 45.03 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.275 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.294 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 18.43 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.313 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.332 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 17.64 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.350 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.369 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 17.76 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.389 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.407 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 18.55 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.440 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.459 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 31.56 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.478 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.543 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 62.42 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.577 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.597 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 31.81 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.618 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.636 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 18.72 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.655 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.682 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 25.7 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.702 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.719 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 16.16 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.739 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.758 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 18.64 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.791 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.810 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 31.68 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.841 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.860 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 29.68 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.879 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.898 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 18.22 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.917 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.005.935 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 17.93 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.029 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 70.79 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.255 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 199.441 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.278 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.351 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 49.87 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.373 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.438 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 43.44 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.460 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.527 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 35.75 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.548 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.596 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 28.5 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.616 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.658 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 21.79 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.677 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.755 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 56.55 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.777 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.844 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 45.93 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.866 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.967 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 78.371 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.006.989 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.067 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 55.94 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.089 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.111 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.253 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 141.89 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.276 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.297 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.430 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 130.95 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.453 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.544 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 70.2 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.566 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.651 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 63.501 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.673 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.692 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.709 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.727 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.745 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.763 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.781 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.798 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.816 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.834 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.852 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.870 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.887 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.913 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.007.932 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.213 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 258.36 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.239 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.262 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.280 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.298 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.417 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 79.37 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.439 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.476 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.557 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 59.5 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.612 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.497 msec. [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.008.970 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.102 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.368 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 201.25 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.501 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 35.25 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.625 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 82.53 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.669 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 16.74 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.725 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 30.95 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.821 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 71.07 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.878 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 31.7 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.927 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 25.59 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.009.995 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 43.33 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.026 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 8.39 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.076 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.638 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.138 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.035 msec. [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.193 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.543 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.580 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.403 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.601 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.620 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.734 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 39.77 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 66.42 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.010.900 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 45.36 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.000 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 74.08 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.023 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.114 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 41.62 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.153 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 15.7 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.177 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 1.61 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.223 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.579 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.279 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.635 msec. [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.321 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.435 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.460 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 1.28 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.517 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 35.04 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.543 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 5.38 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.566 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.79 us [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.608 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 99.43 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.710 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 1.48 us [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.737 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.332 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.794 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.489 msec. [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.839 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.011.920 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.09 msec. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.012.450 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.012.478 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.012.495 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.012.537 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.012.794 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.012.845 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.012.933 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.040 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.118 msec. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.124 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.178 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.204 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.225 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.244 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.267 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.323 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.013.561 [context.cpp:72]619781 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.013.590 [api_impl.cc:3717] 619781 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:50.013.611 [awatchdog_core.c:175](tid:619781) create watchdog for id : 65543, tid : 619781, timeout : 300s [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.013.630 [context.cpp:80]619781 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.013.633 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.013.672 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.013.725 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.013.751 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61025026048, align_size:61025026048 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.015.046 [api_impl.cc:7115] 619781 ReserveMemAddress: device malloc Succ, size=61025026048(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5600000 [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.021.492 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61025026048,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.529 [api_impl.cc:7129] 619781 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.652 [api_impl.cc:7145] 619781 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.734 [device.cpp:143]619781 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.756 [api_impl.cc:2999] 619781 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.769 [device.cc:306] 619781 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.784 [runtime.cc:3393] 619781 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.798 [runtime.cc:3560] 619781 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.812 [api_impl.cc:2987] 619781 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.823 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.837 [api_impl.cc:2992] 619781 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.854 [api_impl.cc:3025] 619781 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.870 [acl.cpp:115]619781 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.881 [acl.cpp:121]619781 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.893 [device.cpp:154]619781 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.021.922 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.939 [context.cpp:174]619781 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.021.958 [api_impl.cc:6913] 619781 CtxSetSysParamOpt: curCtx = 0xaaaac1adc2f0, configOpt=0, configVal=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.021.972 [context.cpp:158]619781 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619419,python3.9):2025-04-03-09:35:50.022.020 [op_base.cc:1214] [619781][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.041 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:50.022.057 [adapter_rts.cc:234] [619781][hrtGetDevice]deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:50.022.082 [op_base.cc:128] [619781][HcclGetDeviceId] deviceLogicId[0] [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.116 [context.cpp:72]619781 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.134 [api_impl.cc:3717] 619781 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.146 [context.cpp:80]619781 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.181 [api_error.cc:1807] 619781 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.204 [memory_task.cc:342] 619781 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.219 [memory_task.cc:473] 619781 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.231 [memory_task.cc:518] 619781 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.248 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.270 [memory_task.cc:1007] 619781 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaaacd6e8af0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.286 [memory_task.cc:1166] 619781 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.299 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.315 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.334 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.351 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.373 [callback.cpp:97]619781 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.408 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.427 [task_info.cc:521] 619781 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.443 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.456 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.472 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.487 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.501 [callback.cpp:109]619781 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.516 [npu_driver.cc:3844] 619610 CqReportGet: cq report get start, device_id=0, tsId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.552 [npu_driver.cc:3851] 619610 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.563 [api_impl.cc:5087] 619610 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.022.582 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.585 [callback.cpp:138]619610 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.022.606 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.022.613 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.022.620 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] DEVICE(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.022.632 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61025026048, align_size:61025026048 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.023.901 [api_impl.cc:7115] 619781 ReserveMemAddress: device malloc Succ, size=61025026048(bytes) start ptr=0x12d100000000, end ptr=0x12df35600000 [INFO] PRE_ACT(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.030.539 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61025026048,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.030.569 [api_impl.cc:7129] 619781 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.030.669 [api_impl.cc:7145] 619781 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.030.785 [device.cpp:231]619781 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.030.804 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.030.824 [acl.cpp:802]619781 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.030.931 [npu_driver.cc:404] 619781 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.030.967 [stream.cc:2246] 619781 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.003 [stream_sqcq_manage.cc:147] 619781 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.019 [npu_driver.cc:882] 619781 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.522 [npu_driver.cc:6125] 619781 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.569 [npu_driver.cc:6137] 619781 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=501, streamId=6, logicCq=501, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.596 [npu_driver.cc:6189] 619781 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=6, logicCq=501. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.615 [npu_driver.cc:5916] 619781 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d85a1000, addr=dfffd85a1000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.631 [stream.cc:653] 619781 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.684 [stream.cc:276] 619781 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.701 [stream.cc:688] 619781 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=0, streamResId=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.031.718 [acl_resource_manager.cpp:814]619781 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.031.732 [acl_resource_manager.cpp:816]619781 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.875 [engine.cc:188] 619781 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.898 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.918 [task_info.cc:409] 619781 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.933 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.945 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.965 [npu_driver.cc:751] 619781 SqTaskSend: device_id=0, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.980 [stars_engine.cc:893] 619781 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.031.996 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.011 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.026 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.043 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.057 [logger.cc:152] 619781 TaskFinished: device_id=0, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.081 [stars_engine.cc:437] 619781 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.097 [stream.cc:2886] 619781 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.114 [stream.cc:852] 619781 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.131 [stream.cc:938] 619781 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.143 [stream.cc:1507] 619781 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.186 [stream_sqcq_manage.cc:451] 619781 FreeLogicCq: Return(0), threadIdentifier(281468852891552), devId(0), tsId(0), cqId(501), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.211 [npu_driver.cc:6163] 619781 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=6, logicCq=501. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.242 [npu_driver.cc:1180] 619781 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=501, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.259 [stream.cc:237] 619781 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.288 [stream_sqcq_manage.cc:216] 619781 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.301 [npu_driver.cc:966] 619781 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.814 [npu_driver.cc:974] 619781 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.833 [stream_sqcq_manage.cc:228] 619781 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.849 [npu_driver.cc:432] 619781 StreamIdFree: stream id free begin, device_id=0, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.867 [npu_driver.cc:441] 619781 StreamIdFree: Stream id free success, stream_id=6, device_id=0, tsId=0, streamFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.032.888 [device_msg_handler.cc:110] 619781 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.033.580 [device_msg_handler.cc:127] 619781 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.033.636 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.033.670 [npu_driver.cc:500] 619781 GetFaultEvent: get fault event, device_id=0, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.033.754 [acl.cpp:782]619781 GetFaultEventInfo: can not get fault event of device 0, runtime errorCode is 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:50.033.784 [error_manager.cc:989]619781 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.034.132 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.034.172 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.034.195 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.034.214 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.034.503 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.034.595 [context.cpp:72]619781 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.619 [api_impl.cc:3717] 619781 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.034.637 [context.cpp:80]619781 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.658 [api_error.cc:1807] 619781 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.034.662 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.676 [memory_task.cc:342] 619781 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.704 [memory_task.cc:473] 619781 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.717 [memory_task.cc:518] 619781 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.731 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.748 [memory_task.cc:1007] 619781 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaaacd6e9510, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.764 [memory_task.cc:1166] 619781 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.776 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.789 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.808 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.823 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.034.842 [callback.cpp:97]619781 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.861 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.877 [task_info.cc:521] 619781 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.890 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.903 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.920 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.937 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.034.948 [callback.cpp:109]619781 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.952 [npu_driver.cc:3844] 619610 CqReportGet: cq report get start, device_id=0, tsId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.970 [npu_driver.cc:3851] 619610 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.034.981 [api_impl.cc:5087] 619610 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.034.999 [callback.cpp:138]619610 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.035.016 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.035.016 [device.cpp:231]619781 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.029 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.037 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.035.065 [acl.cpp:802]619781 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.096 [npu_driver.cc:404] 619781 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.125 [stream.cc:2246] 619781 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.162 [stream_sqcq_manage.cc:147] 619781 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.179 [npu_driver.cc:882] 619781 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.414 [npu_driver.cc:6125] 619781 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.454 [npu_driver.cc:6137] 619781 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=502, streamId=7, logicCq=502, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.475 [npu_driver.cc:6189] 619781 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=7, logicCq=502. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.491 [npu_driver.cc:5916] 619781 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d85a1000, addr=dfffd85a1000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.507 [stream.cc:653] 619781 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.543 [stream.cc:276] 619781 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.559 [stream.cc:688] 619781 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=0, streamResId=65534 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.035.576 [acl_resource_manager.cpp:814]619781 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.035.589 [acl_resource_manager.cpp:816]619781 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.631 [engine.cc:188] 619781 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.649 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.665 [task_info.cc:409] 619781 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.679 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.692 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.710 [npu_driver.cc:751] 619781 SqTaskSend: device_id=0, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.723 [stars_engine.cc:893] 619781 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.739 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.753 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.768 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.781 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.808 [logger.cc:152] 619781 TaskFinished: device_id=0, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.825 [stars_engine.cc:437] 619781 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.841 [stream.cc:2886] 619781 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.853 [stream.cc:852] 619781 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.867 [stream.cc:938] 619781 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.881 [stream.cc:1507] 619781 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.899 [stream_sqcq_manage.cc:451] 619781 FreeLogicCq: Return(0), threadIdentifier(281468852891552), devId(0), tsId(0), cqId(502), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.919 [npu_driver.cc:6163] 619781 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=7, logicCq=502. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.945 [npu_driver.cc:1180] 619781 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=502, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.960 [stream.cc:237] 619781 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.971 [stream_sqcq_manage.cc:216] 619781 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.035.982 [npu_driver.cc:966] 619781 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.422 [npu_driver.cc:974] 619781 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.440 [stream_sqcq_manage.cc:228] 619781 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.458 [npu_driver.cc:432] 619781 StreamIdFree: stream id free begin, device_id=0, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.475 [npu_driver.cc:441] 619781 StreamIdFree: Stream id free success, stream_id=7, device_id=0, tsId=0, streamFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.492 [device_msg_handler.cc:110] 619781 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.808 [device_msg_handler.cc:127] 619781 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.839 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.036.861 [npu_driver.cc:500] 619781 GetFaultEvent: get fault event, device_id=0, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.036.915 [acl.cpp:782]619781 GetFaultEventInfo: can not get fault event of device 0, runtime errorCode is 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:50.036.933 [error_manager.cc:989]619781 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.037.221 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.037.253 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.037.273 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.037.301 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.037.371 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.447 [context.cpp:72]619781 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.469 [api_impl.cc:3717] 619781 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.488 [context.cpp:80]619781 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.511 [api_error.cc:1807] 619781 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.531 [memory_task.cc:342] 619781 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.547 [memory_task.cc:473] 619781 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.559 [memory_task.cc:518] 619781 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.572 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.589 [memory_task.cc:1007] 619781 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaaacd6e9f80, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.604 [memory_task.cc:1166] 619781 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.616 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.631 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.651 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.665 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.685 [callback.cpp:97]619781 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.704 [task_res.cc:380] 619781 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.723 [task_info.cc:521] 619781 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.737 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.752 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.779 [task_submit.cc:573] 619781 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.793 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.804 [callback.cpp:109]619781 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.808 [npu_driver.cc:3844] 619610 CqReportGet: cq report get start, device_id=0, tsId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.822 [npu_driver.cc:3851] 619610 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.833 [api_impl.cc:5087] 619610 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.847 [callback.cpp:138]619610 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.863 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.868 [device.cpp:231]619781 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.876 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.883 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.037.898 [acl.cpp:802]619781 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.930 [npu_driver.cc:404] 619781 StreamIdAlloc: stream id alloc begin, device_id=0, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.957 [stream.cc:2246] 619781 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.037.994 [stream_sqcq_manage.cc:147] 619781 Alloc: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.009 [npu_driver.cc:882] 619781 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.242 [npu_driver.cc:6125] 619781 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.281 [npu_driver.cc:6137] 619781 LogicCqAllocateV2: success: deviceId=0, tsId=0, cqId=503, streamId=8, logicCq=503, cqeDepth=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.302 [npu_driver.cc:6189] 619781 StreamBindLogicCq: success: deviceId=0, tsId=0, streamId=8, logicCq=503. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.318 [npu_driver.cc:5916] 619781 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=d85a1000, addr=dfffd85a1000size=4096 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.332 [stream.cc:653] 619781 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.370 [stream.cc:276] 619781 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.386 [stream.cc:688] 619781 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=0, streamResId=65534 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.038.401 [acl_resource_manager.cpp:814]619781 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.038.415 [acl_resource_manager.cpp:816]619781 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.456 [engine.cc:188] 619781 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.484 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.499 [task_info.cc:409] 619781 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.513 [stream.cc:1400] 619781 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.525 [stream.cc:2595] 619781 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.543 [npu_driver.cc:751] 619781 SqTaskSend: device_id=0, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.556 [stars_engine.cc:893] 619781 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.573 [stream.cc:1494] 619781 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.585 [npu_driver.cc:5547] 619781 GetSqHead: deviceId=0, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.599 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.614 [task.cc:185] 619781 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.626 [logger.cc:152] 619781 TaskFinished: device_id=0, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.643 [stars_engine.cc:437] 619781 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.658 [stream.cc:2886] 619781 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.670 [stream.cc:852] 619781 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.683 [stream.cc:938] 619781 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.695 [stream.cc:1507] 619781 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.715 [stream_sqcq_manage.cc:451] 619781 FreeLogicCq: Return(0), threadIdentifier(281468852891552), devId(0), tsId(0), cqId(503), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.735 [npu_driver.cc:6163] 619781 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=8, logicCq=503. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.761 [npu_driver.cc:1180] 619781 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=503, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.776 [stream.cc:237] 619781 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.787 [stream_sqcq_manage.cc:216] 619781 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.038.799 [npu_driver.cc:966] 619781 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.238 [npu_driver.cc:974] 619781 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.256 [stream_sqcq_manage.cc:228] 619781 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.269 [npu_driver.cc:432] 619781 StreamIdFree: stream id free begin, device_id=0, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.294 [npu_driver.cc:441] 619781 StreamIdFree: Stream id free success, stream_id=8, device_id=0, tsId=0, streamFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.311 [device_msg_handler.cc:110] 619781 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.624 [device_msg_handler.cc:127] 619781 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.657 [runtime.cc:5239] 619781 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.039.678 [npu_driver.cc:500] 619781 GetFaultEvent: get fault event, device_id=0, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.039.730 [acl.cpp:782]619781 GetFaultEventInfo: can not get fault event of device 0, runtime errorCode is 0 [INFO] GE(619419,python3.9):2025-04-03-09:35:50.039.747 [error_manager.cc:989]619781 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.040.035 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.040.067 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.040.086 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.040.103 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619419,fffe92fdefa0,python3.9):2025-04-03-09:35:50.040.184 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:50.075.012 [awatchdog_monitor.c:160](tid:619459) thread 619755 not exist [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.244.175 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.244.209 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.244.226 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.244.242 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.244.293 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.244.306 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.244.318 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.564.181 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.564.220 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.564.235 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.564.272 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.564.299 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.564.317 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.564.342 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.589.651 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.687 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.589.710 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.589.760 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.783 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.807 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.877 [stars_engine.cc:643] 619419 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.908 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.927 [task_manager.cc:485] 619419 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.949 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.967 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.589.984 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.005 [stars_engine.cc:643] 619419 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.022 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.038 [task_manager.cc:485] 619419 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.050 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.067 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.080 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.100 [stars_engine.cc:643] 619419 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.116 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.143 [task_manager.cc:485] 619419 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.156 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.172 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.186 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.210 [stars_engine.cc:622] 619419 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=497, retCode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.226 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.240 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.590.255 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.590.275 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.295 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.311 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.328 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.344 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.359 [stream.cc:2886] 619419 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.371 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.385 [stream.cc:2886] 619419 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.397 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.411 [stream.cc:2886] 619419 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.424 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.437 [stream.cc:2886] 619419 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.449 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.463 [stream.cc:2886] 619419 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.474 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.488 [stream.cc:2886] 619419 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.500 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.514 [stream.cc:2886] 619419 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.535 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.547 [stream.cc:2886] 619419 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.562 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.574 [stream.cc:2886] 619419 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.587 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.600 [stream.cc:2886] 619419 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.616 [event.cc:310] 619419 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.644 [task_res.cc:380] 619419 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=0, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.669 [event.cc:249] 619419 UpdateLatestRecord: device_id=0, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.689 [event_task.cc:259] 619419 ConstructSqeForEventRecordTask: event_record: device_id=0, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.707 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.723 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.743 [task_submit.cc:573] 619419 AllocTaskAndSendStars: device_id=0, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.759 [stars_engine.cc:1122] 619419 SyncTask: Task Wait: device_id=0, stream_id=3, task_id=0, logicCqId=498. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.783 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.798 [stars_engine.cc:1071] 619419 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.813 [stars_engine.cc:1017] 619419 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.832 [stars_engine.cc:1027] 619419 ProcReport: taskType=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.844 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.861 [event.cc:249] 619419 UpdateLatestRecord: device_id=0, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.879 [event_task.cc:211] 619419 DoCompleteSuccessForEventRecordTask: event_record: device_id=0, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea55ec5725 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.897 [event.cc:122] 619419 TryFreeEventIdAndCheckCanBeDelete: device_id=0, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.914 [task_res.cc:323] 619419 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=0, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.944 [npu_driver.cc:5618] 619419 GetSqEnable: deviceId=0, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.959 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.590.971 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.590.985 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.065 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.084 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.101 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.118 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.135 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.147 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.162 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.177 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.189 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.204 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.223 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.236 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.263 [stars_engine.cc:622] 619419 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=498, retCode=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.279 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.591.291 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.304 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.591.319 [memory.cpp:421]619419 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.39s ============================================================================================================== ] [619755] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340575] [debug] [619755] [lcal_comm.cpp:557] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340826] [debug] [619755] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340837] [debug] [619755] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340847] [debug] [619755] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340855] [debug] [619755] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340864] [debug] [619755] [lcal_comm.cpp:515] rank 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340872] [debug] [619755] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340880] [debug] [619755] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340888] [debug] [619755] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340896] [debug] [619755] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360326] [debug] [619755] [lcal_comm.cpp:254] InitCommMem 0/8, localRank_ : 0, localRankSize_ : 8 success [2025-04-03 09:35:45.360448] [info] [619755] [lcal_comm.cpp:259] LcalCommInit 0/8 success and extraFlag:0 commArgs_.localRank : 0 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.30745] [error] [619781] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.34993] [error] [619781] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.37848] [error] [619781] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.374 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.417 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:50.600.501 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619419:281473133236256,MainProcess):2025-04-03-09:35:50.600.665 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.750 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.840 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.889 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.600.917 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.177 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.180 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.186 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.208 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.238 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.241 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.254 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.254 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.256 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.276 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.301 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.305 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.320 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.326 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.330 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.341 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.351 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.353 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.366 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.374 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.375 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.380 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.406 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.408 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.418 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.420 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.429 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.438 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.440 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.441 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.447 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.451 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.460 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.471 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.480 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.486 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.491 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.496 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.756.500 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.506 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.518 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.756.532 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:50.756.950 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:50.757.027 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.757.044 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.757.064 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.757.089 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.757.100 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.757.113 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.250 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.368 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619419,ffff31897fa0,python3.9):2025-04-03-09:35:50.850.478 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.653 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.679 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619419,ffff320a7fa0,python3.9):2025-04-03-09:35:50.850.716 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.790 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.809 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.850.883 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619419,ffff30877fa0,python3.9):2025-04-03-09:35:50.850.986 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.061 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.081 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619419,ffff31087fa0,python3.9):2025-04-03-09:35:50.851.162 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.358 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.384 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.409 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.851.430 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.582 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.609 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.628 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.663 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.683 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.715 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.733 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.750 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.765 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.783 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.802 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.819 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.835 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.851 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.863 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.886 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.905 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.917 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.932 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.944 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.958 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.851.968 [stream.cpp:131]619419 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.985 [api_impl.cc:1394] 619419 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.851.998 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.852.013 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.852.025 [api_error.cc:895] 619419 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.852.039 [stream.cpp:146]619419 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.091 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.246 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.320 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.341 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.369 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.594 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.852.616 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.466 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.497 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.516 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.534 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.554 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.855 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.878 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.895 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.963 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.853.983 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.854.003 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_0 [INFO] GE_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.863.654 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.863.690 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.863.721 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.863.737 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.760 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.863.777 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.863.793 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.810 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.863.822 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.863.852 [device.cpp:143]619419 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.873 [api_impl.cc:2999] 619419 SetDevice: device_id=0, deviceMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.899 [device.cc:306] 619419 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.913 [runtime.cc:3393] 619419 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.933 [runtime.cc:3560] 619419 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.948 [api_impl.cc:2987] 619419 NewDevice: SetCurRef = 0xaaaac1adc2f0,userDeviceId=0,deviceId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.962 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.863.983 [api_impl.cc:2992] 619419 NewDevice: New device success, user_device_id=0, die_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:50.864.002 [api_impl.cc:3025] 619419 SetDevice: SetDevice success, curCtx=0xaaaac1adc2f0, user_device_id=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.864.019 [acl.cpp:115]619419 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.864.034 [acl.cpp:121]619419 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:50.864.052 [device.cpp:154]619419 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 0 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:50.864.080 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:51.033.582 [awatchdog_monitor.c:160](tid:619459) thread 619781 not exist [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.076.171 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.076.197 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.076.213 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.076.228 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.076.242 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.076.257 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.076.275 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.180.740 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.180.769 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.180.788 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.180.802 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.180.828 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.180.909 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.268.174 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.268.209 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.268.228 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.268.246 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.268.260 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.268.270 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.268.281 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.169 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.168 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.170 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.171 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.172 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.178 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.178 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.588.195 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.254 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.268 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.270 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.281 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.287 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.293 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.297 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.303 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.309 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.313 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.316 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.331 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.331 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.344 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.352 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.357 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.361 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.362 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.365 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.374 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.380 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.385 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.393 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.392 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.400 [tensor_data_transfer.cpp:1054]619735 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.408 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.419 [tensor_data_transfer.cpp:1056]619735 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.423 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.416 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.425 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.431 [api_impl.cc:6121] 619735 MemQueuePeek: Start to peek queue on devId 0, qid is 0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.439 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.444 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.452 [npu_driver.cc:4330] 619735 MemQueuePeek: peek mem queue, deviceId=0, qid=0, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.458 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.462 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.588.484 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.488 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.497 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.501 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.588.510 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.780.198 [trs_cb_event.c:119][ascend][curpid:619419,619611][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =65; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:51.780.219 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.780.236 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:51.780.249 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.780.260 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:51.780.275 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.780.288 [npu_driver.cc:3828] 619611 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=65. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.539 [op_base.cc:2136] [619419]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] DEVICE(619419,fffe927cefa0,python3.9):2025-04-03-09:35:51.984.558 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.984.577 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.595 [adapter_rts.cc:198] [619419][hrtGetDeviceRefresh]deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.616 [op_base.cc:2151] [619419][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.652 [op_base.cc:128] [619419][HcclGetDeviceId] deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.669 [op_base.cc:2168] [619419]com is not global com [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.685 [hcom_plugin.cc:381] [619419][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.737 [hccl_comm.cc:1375] [619419]UnRegistTaskAbortHandler begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.755 [task_abort_handler.cc:140] [619419]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.984.775 [api_impl.cc:4192] 619419 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.984.793 [runtime.cc:4698] 619419 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.809 [hccl_communicator.cc:2112] [619419][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.984.839 [hccl_communicator.cc:2114] [619419][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.852 [hccl_communicator.cc:2082] [619419]start to stop the backGround Thread [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.984.870 [heartbeat.cc:1203] [619419]Heartbeat has been destroyed [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.886 [heartbeat.cc:451] [619419][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.984.903 [heartbeat.cc:452] [619419]Heartbeat has been destroyed [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.919 [hccl_impl.cc:69] [619419]start hccl resources destruction:deviceLogicId[0] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.984.939 [heartbeat.cc:1203] [619419]Heartbeat has been destroyed [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.984.954 [heartbeat.cc:451] [619419][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.984.966 [heartbeat.cc:452] [619419]Heartbeat has been destroyed [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.985.077 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.985.237 [task_exception_handler.cc:1232] [619419]deInit taskFailCallback [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.985.730 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.985.754 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.985.796 [api_error.cc:1278] 619419 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:51.985.814 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.985.849 [hccl_socket_manager.cc:81] [619419][DeInit][Server]ip[192.168.2.199] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.985.867 [hccl_socket_manager.cc:83] [619419][DeInit][Server]ip[192.168.2.199] port[16666] [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.985.881 [hccl_socket.cc:232] [619419][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.985.896 [network_manager.cc:69] [619419]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.985.910 [network_manager.cc:458] [619419]NetworkManager devicePhyId_[0] vnic stopped. refcount[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:51.985.933 [ra_host.c:911]tid:619419,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[0], local_ip[192.168.2.199] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:51.986.290 [ra_host.c:404]tid:619419,ra_socket_deinit(404) : Input parameters: phy_id[0] family[2] local_ip[192.168.2.199] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.986.442 [hccl_socket.cc:68] [619419][HcclSocket][DeInit] device[0] stops listen on ip[192.168.2.199], port[16666] success, socketType[0]. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:51.986.463 [hccl_socket.cc:232] [619419][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.986.489 [hccl_communicator.cc:1741] [619419][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.986.506 [network_manager.cc:69] [619419]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:51.986.520 [ra_host.c:248]tid:619419,ra_deinit(248) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:51.986.537 [ra_hdc.c:651]tid:619419,ra_hdc_deinit(651) : hdc deinit start! phy_id[0] [INFO] DRV(619419,python3.9):2025-04-03-09:35:51.986.643 [hdc_client.c:417][ascend][curpid:619419,619419][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4270) [INFO] ROCE(619419,python3.9):2025-04-03-09:35:51.986.700 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:51.986.717 [ra_hdc.c:685]tid:619419,ra_hdc_deinit(685) : hdc deinit OK! phy_id[0] [INFO] ROCE(619419,python3.9):2025-04-03-09:35:51.986.730 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.789 [tsd_client.cpp:189][TsdCapabilityGet][tid:619419] TsdCapabilityGet Begin. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.806 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.818 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.834 [process_mode_manager.cpp:1042][CapabilityGet][tid:619419] [ProcessModeManager] enter into CapabilityGet process deviceId[0] type[6] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:51.986.851 [adapter_tdt.cc:66] [619419]Get TsdCapability success. deviceLogicId[0] [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.866 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.881 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.893 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619419] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.913 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619419] add close subproc:366, proctype:0 [INFO] TDT(619419,python3.9):2025-04-03-09:35:51.986.957 [hdc_client.cpp:154][TsdRecvData][tid:619419] recv timeout is:150000 ms [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.058.846 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619419] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.058.866 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619419] [TsdClient] DeviceMsgProc recvMsg realDeviceId[0] msgType[61] localDevId[0] rspCode[0]heterogeneousSubPid[366], tsdSupportLevel_[63] [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.058.886 [hdc_client.cpp:172][TsdRecvData][tid:619419] [deviceId=0] the Tsd recv data already [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.058.901 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619419] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.058.918 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619419] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.058.932 [adapter_tdt.cc:52] [619419]Open TsdProcess success. deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.058.947 [network_manager.cc:405] [619419][DeInit]finish CloseHccpProcess, phy_id[0], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.058.965 [network_manager.cc:409] [619419]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.058.977 [hccl_communicator_attrs.cc:1307] [619419][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.058.993 [hccl_communicator_attrs.cc:1275] [619419]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.014 [notify_pool_impl.cc:43] [619419]NotifyPoolImpl Destroy. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.039 [notify_pool_impl.cc:56] [619419]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.072 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.088 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.100 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.117 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.131 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.142 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.156 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.171 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.182 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.195 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.208 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.222 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.243 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.259 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.277 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=1. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.300 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=1.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.317 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.328 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.341 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.355 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.366 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.379 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.395 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.410 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=2. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.424 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=2.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.439 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.450 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.463 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.483 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.494 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.508 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.523 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.537 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=3. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.552 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=3.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.568 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.579 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.591 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.605 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.616 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.629 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.644 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.658 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=4. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.673 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=4.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.688 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.699 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.712 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.724 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.738 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.750 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.763 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.777 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=5. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.791 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=5.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.807 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.817 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.830 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.843 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.854 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.868 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.887 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.898 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=6. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.059.914 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=6.) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.930 [api_impl.cc:3100] 619419 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.940 [api_impl.cc:3111] 619419 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.952 [api_impl.cc:3130] 619419 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.059.966 [p2p_mgmt.cc:171] [619419]local logic id:0, local physic id:0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.977 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.059.990 [api_impl.cc:3520] 619419 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.060.005 [p2p_mgmt.cc:178] [619419]disable p2p: local logic id:0, remote physic id:7. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.060.019 [api_impl.cc:3184] 619419 DisableP2P: Disable P2P devIdDes=0, phyIdSrc=7. [INFO] DRV(619419,python3.9):2025-04-03-09:35:52.060.034 [devdrv_pcie.c:890][ascend][curpid:619419,619419][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=0; peer_phy_devid=7.) [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.060.050 [hccl_communicator.cc:2043] [619419]Profiling is deinitiated. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.060.066 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.060.089 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.060.107 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.060.125 [opexecounter.cc:103] [619419]free counter mem resource [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.060.141 [atrace_api.c:118](tid:619419) AtraceDestroy start. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.060.172 [tracer_mgr_operate.c:248](tid:619419) destroy object HCCL_619755_0, exitSave(true). [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.060.192 [atrace_api.c:120](tid:619419) AtraceDestroy end. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.060.203 [adapter_trace.cc:33] [619419]Call TraceDestroy, Params:handle[0xaaaab0f64a48] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.061.917 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.061.954 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.061.982 [queue_notify_manager.cc:100] [619419]QueueNotifyManager Destroy. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.000 [queue_notify_manager.cc:106] [619419]QueueNotifyManager Destroy success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.127 [queue_notify_manager.cc:100] [619419]QueueNotifyManager Destroy. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.143 [queue_notify_manager.cc:106] [619419]QueueNotifyManager Destroy success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.211 [ccl_buffer_manager.cc:125] [619419][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.231 [ccl_buffer_manager.cc:147] [619419][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.248 [ccl_buffer_manager.cc:160] [619419][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.272 [ccl_buffer_manager.cc:163] [619419][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.286 [ccl_buffer_manager.cc:166] [619419][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.332 [op_base.cc:128] [619419][HcclGetDeviceId] deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.396 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe880035a0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.419 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[1] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.459 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:61, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.550 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [137]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.569 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88003330] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.587 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[2] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.604 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:60, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.652 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [67]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.669 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88003a80] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.682 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[3] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.698 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:63, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.733 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [53]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.750 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe880030c0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.764 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[4] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.781 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:59, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.820 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [57]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.836 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88002e50] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.849 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[5] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.865 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:58, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.903 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [55]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.920 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88003810] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.932 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[6] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.062.948 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:62, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.062.996 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [64]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.012 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88002be0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.027 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[7] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.043 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:57, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.077 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [51]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.093 [adapter_hccp.cc:1067] [619419]ra socket batch close fdhandle[0xfffe88002970] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.110 [ra_host.c:796]tid:619419,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[0], local_ip[192.168.0.44], cnt[8] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.126 [rs_socket.c:1937]tid:619419,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:56, state:9 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.159 [adapter_hccp.cc:1099] [619419]ra socket batch close success,take time [50]us [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.179 [topoinfo_detect.cc:555] [619419]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.195 [network_manager.cc:69] [619419]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.211 [ra_host.c:404]tid:619419,ra_socket_deinit(404) : Input parameters: phy_id[0] family[2] local_ip[192.168.0.44] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.228 [rs.c:1629]tid:619419,rs_socket_deinit(1629) : rs socket deinit start, phy_id:0 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.245 [rs.c:1641]tid:619419,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.266 [rs.c:1653]tid:619419,rs_socket_deinit(1653) : socket deinit success, phy_id:0, local_ip:192.168.0.44 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.283 [network_manager.cc:1233] [619419]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.063.297 [network_manager.cc:69] [619419]NetworkManager::GetInstance deviceLogicID[0]. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.312 [ra_host.c:248]tid:619419,ra_deinit(248) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.328 [ra_peer.c:821]tid:619419,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[0] start [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.354 [rs.c:1783]tid:619419,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619419,python3.9):2025-04-03-09:35:52.063.368 [rs_epoll.c:523]tid:619752,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:52 [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.063.396 [rs_epoll.c:544]tid:619752,rs_epoll_handle(544) : QUIT thread_id:281469423054752, pid:619419, read num:8 [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.100.180 [trs_cb_event.c:119][ascend][curpid:619419,619610][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=0, tid =64; ret=16) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.100.203 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.100.216 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.100.230 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.100.242 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.100.267 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.100.281 [npu_driver.cc:3828] 619610 CqReportIrqWait: Irq wait start, deviceId=0, tsId=0, groupId=64. [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.144.513 [rs_epoll.c:679]tid:619753,rs_connect_handle(679) : QUIT thread_id:281468869799840, pid:619419 [INFO] ROCE(619419,python3.9):2025-04-03-09:35:52.163.669 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.163.691 [rs.c:1810]tid:619419,rs_deinit(1810) : rs_deinit chip_id[0] ok [INFO] ROCE(619419,python3.9):2025-04-03-09:35:52.163.709 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619419,python3.9):2025-04-03-09:35:52.163.723 [ra_peer.c:852]tid:619419,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[0] succ [INFO] ROCE(619419,python3.9):2025-04-03-09:35:52.163.738 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.163.760 [network_manager.cc:409] [619419]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.163.775 [topoinfo_detect.cc:219] [619419]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619419,python3.9):2025-04-03-09:35:52.163.789 [hccl_socket.cc:232] [619419][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.163.813 [op_base.cc:2196] [619419]op_base comm destroy complete, take time [179236]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[0]. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.163.833 [device.cpp:183]619419 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.163.854 [api_impl.cc:3419] 619419 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.163.870 [runtime.cc:3444] 619419 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.163.886 [api_impl.cc:3449] 619419 DeviceReset: Succ, userDeviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.163.898 [device.cpp:189]619419 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 0. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.163.939 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.163.965 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.163.980 [hcom_executor.cc:40] [619419]Hcom Excutor Finalize start. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.143 [api_impl.cc:3419] 619749 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.140 [api_impl.cc:3419] 619747 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.192 [runtime.cc:3444] 619749 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.204 [runtime.cc:3444] 619747 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.211 [api_impl.cc:3449] 619749 DeviceReset: Succ, userDeviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.219 [api_impl.cc:3449] 619747 DeviceReset: Succ, userDeviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.384 [api_impl.cc:3419] 619748 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.422 [runtime.cc:3444] 619748 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.432 [api_impl.cc:3419] 619750 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.438 [api_impl.cc:3449] 619748 DeviceReset: Succ, userDeviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.457 [runtime.cc:3444] 619750 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.472 [api_impl.cc:3449] 619750 DeviceReset: Succ, userDeviceId=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.581 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.164.603 [adapter_rts.cc:321] [619419][hrtCtxSetCurrent] success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.164.619 [stream.cc:182] [619419]Switch Ctx ret[0], curCtx[0xaaaac1adc2f0], setCtx[0xaaaac1adc2f0], stream id[5] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.639 [api_impl.cc:1228] 619419 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.164.662 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.164.701 [single_op_manager.cc:50]619419 ReleaseResource:ReleaseResource in. resource id = 0xaaaaccf65be0 [INFO] GE(619419,python3.9):2025-04-03-09:35:52.164.726 [caching_allocator.cc:402]619419 TryFreeBlocks:Try free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.164.742 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:52.164.772 [npu_memory_allocator.cc:72]619419 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.164.788 [stream_executor.cpp:197]619419 RemoveExecutor: To remove executor by stream = 187650559859680 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.164.822 [acl_resource_manager.cpp:823]619419 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.840 [stream.cc:852] 619419 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.859 [stream.cc:938] 619419 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.874 [stream.cc:1507] 619419 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.907 [stream_sqcq_manage.cc:451] 619419 FreeLogicCq: Return(0), threadIdentifier(281473133236256), devId(0), tsId(0), cqId(500), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.934 [npu_driver.cc:6163] 619419 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=5, logicCq=500. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.164.974 [npu_driver.cc:1180] 619419 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=500, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.166.756 [stream.cc:237] 619419 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.166.782 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.166.798 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.167.395 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.167.415 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.167.433 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.167.463 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=5, device_id=0, tsId=0, streamFlag=1536 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.168.146 [stream.cc:185] [619419][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaaccf65be0], stream id[5] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.179 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.168.200 [adapter_rts.cc:321] [619419][hrtCtxSetCurrent] success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.168.214 [stream.cc:193] [619419]Restore Ctx ret[0], setCtx[0xaaaac1adc2f0], stream id[5] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.268 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.168.286 [adapter_rts.cc:321] [619419][hrtCtxSetCurrent] success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.168.298 [stream.cc:182] [619419]Switch Ctx ret[0], curCtx[0xaaaac1adc2f0], setCtx[0xaaaac1adc2f0], stream id[4] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.316 [api_impl.cc:1228] 619419 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.168.330 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.168.345 [single_op_manager.cc:50]619419 ReleaseResource:ReleaseResource in. resource id = 0xaaaaccee9b80 [INFO] GE(619419,python3.9):2025-04-03-09:35:52.168.364 [caching_allocator.cc:402]619419 TryFreeBlocks:Try free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.168.376 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:52.168.388 [npu_memory_allocator.cc:72]619419 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.168.401 [stream_executor.cpp:197]619419 RemoveExecutor: To remove executor by stream = 187650559351680 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.168.413 [acl_resource_manager.cpp:823]619419 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.427 [stream.cc:852] 619419 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.443 [stream.cc:938] 619419 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.454 [stream.cc:1507] 619419 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.476 [stream_sqcq_manage.cc:451] 619419 FreeLogicCq: Return(0), threadIdentifier(281473133236256), devId(0), tsId(0), cqId(499), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.496 [npu_driver.cc:6163] 619419 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=4, logicCq=499. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.168.528 [npu_driver.cc:1180] 619419 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=499, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.169.984 [stream.cc:237] 619419 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.169.999 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.170.011 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.170.481 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.170.500 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.170.526 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.170.543 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=4, device_id=0, tsId=0, streamFlag=1536 [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.155 [stream.cc:185] [619419][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaaccee9b80], stream id[4] [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.171.174 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.189 [adapter_rts.cc:321] [619419][hrtCtxSetCurrent] success [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.203 [stream.cc:193] [619419]Restore Ctx ret[0], setCtx[0xaaaac1adc2f0], stream id[4] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.234 [hcom_executor.cc:45] [619419][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.171.269 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.171.294 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.321 [hcom_ops_kernel_info_store.cc:3423] [619419]finalize hccl kernel info store. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.348 [hcom_common.cc:85] [619419][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[0] [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.370 [plugin_manager.cc:56] [619419]hccl ops plugin finalize start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.387 [hcom_plugin.cc:90] [619419]Finalize start. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.453 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.469 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.484 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.498 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.526 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.541 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.551 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.563 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.585 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.600 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.611 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.622 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.644 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.658 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.669 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.680 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.702 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.724 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.735 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.747 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.770 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.784 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.794 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.806 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.829 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.843 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.853 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.865 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.887 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.902 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.911 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.923 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.945 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.959 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.968 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.171.981 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.003 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.017 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.027 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.039 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.062 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.076 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.086 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.099 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.121 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.135 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.145 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.170 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.197 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.212 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.221 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.246 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.292 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.307 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.317 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.329 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.351 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.365 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.376 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.387 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.410 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.424 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.435 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.446 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.458 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.469 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.480 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.492 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.504 [hcom_plugin.cc:119] [619419]Finalize: HcomDestroy success. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.518 [plugin_manager.cc:62] [619419]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.172.588 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.172.609 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.622 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.638 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.649 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.661 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.674 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.686 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.696 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.708 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.718 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.731 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.744 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.760 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.771 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.784 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.794 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.807 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.820 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.831 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.843 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.856 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.866 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.878 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.889 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.903 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.913 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.925 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.936 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.947 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.959 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.972 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.982 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.172.993 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.005 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.018 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.029 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.040 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.052 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.066 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.076 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.087 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.099 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.112 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.122 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.134 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.145 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.171 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.182 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.194 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.207 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.220 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.231 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.245 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.257 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.269 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.282 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.293 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.304 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.318 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.328 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.339 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.351 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.363 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.374 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.387 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.397 [hcom_common.cc:958] [619419][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.409 [hcom_common.cc:961] [619419]Entry-HcomDestroy:void skip [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.423 [hcom.cc:337] [619419][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:52.173.433 [hcom.cc:340] [619419]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.459 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.480 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.667 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.689 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.706 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.723 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.746 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.764 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.780 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61025026048 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.794 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.809 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61025025536, total count : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.826 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.840 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.856 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.869 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.882 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61025026048 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.898 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.913 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61025025536, total count : 1 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.927 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116396.00MB,"idle mem":116395.99MB,"used by event mem":0.00MB,"eager free mem":116395.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.955 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116396.00MB,"idle mem":116395.99MB,"used by event mem":0.00MB,"eager free mem":116395.99MB}. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.975 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.173.995 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.019 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.038 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.057 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.075 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.092 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.110 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.129 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.145 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.160 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.176 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61025026048,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.193 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.210 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.231 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe7c079c10] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.250 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe7c07dc00] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.174.276 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58198M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.300 [callback.cpp:37]619419 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.321 [callback.cpp:44]619419 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.360 [context.cpp:72]619419 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.383 [api_impl.cc:3717] 619419 ContextSetCurrent: set context=aaaac1adc2f0, flag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.397 [context.cpp:80]619419 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.414 [stream.cpp:75]619419 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619419,python3.9):2025-04-03-09:35:52.174.440 [stream_manage_func_registry.cc:22]619419 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.456 [stream.cpp:80]619419 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.468 [api_impl.cc:1228] 619419 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.485 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.174.500 [single_op_manager.cc:50]619419 ReleaseResource:ReleaseResource in. resource id = 0xaaaac2b2c890 [INFO] GE(619419,python3.9):2025-04-03-09:35:52.174.519 [caching_allocator.cc:402]619419 TryFreeBlocks:Try free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.174.531 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:52.174.545 [npu_memory_allocator.cc:72]619419 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.555 [stream_executor.cpp:197]619419 RemoveExecutor: To remove executor by stream = 187650387658896 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.568 [acl_resource_manager.cpp:823]619419 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.581 [stream.cc:852] 619419 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.597 [stream.cc:938] 619419 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.612 [event.cc:1036] 619419 WaitForBusy: device_id=0, event_id=0 finished map already empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.630 [npu_driver.cc:626] 619419 EventIdFree: event id free begin, device_id=0, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.655 [raw_device.cc:1158] 619419 FreeEventIdFromDrv: device_id=0, event_id=0 tear down. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.670 [event.cc:61] 619419 ~Event: device_id=0, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.688 [npu_driver.cc:3803] 619419 SqCqFree: deviceId=0, tsId=0, sqId=135, cqId=135 [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.174.725 [trs_sqcq.c:1491][ascend][curpid:619419,619610][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.725 [subscribe.cc:244] 619419 Delete: delete callback subscribe: groupId=64, threadId=281471259520928, devId=0, tsId=0, streamId=2, sqId=135, cqId=135 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.745 [npu_driver.cc:3833] 619610 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=64, drvRetCode=16. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.753 [stream.cc:1507] 619419 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.758 [api_impl.cc:5058] 619610 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.772 [callback.cpp:132]619610 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.776 [stream_sqcq_manage.cc:451] 619419 FreeLogicCq: Return(0), threadIdentifier(281473133236256), devId(0), tsId(0), cqId(497), isFastCq(0). [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.794 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.801 [npu_driver.cc:6163] 619419 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=2, logicCq=497. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.807 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.819 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.834 [npu_driver.cc:1180] 619419 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=497, drvFlag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.837 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.855 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.870 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.174.872 [awatchdog_monitor.c:160](tid:619459) thread 619747 not exist [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.880 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.174.899 [awatchdog_monitor.c:160](tid:619459) thread 619749 not exist [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.900 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.174.924 [awatchdog_monitor.c:160](tid:619459) thread 619750 not exist [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.924 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:52.174.945 [awatchdog_monitor.c:160](tid:619459) thread 619748 not exist [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.947 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.174.967 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.982 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.174.993 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.005 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.017 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.030 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.041 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.053 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.065 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.084 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.095 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.108 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.121 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.132 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.143 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.157 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.167 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.179 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.193 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.204 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.214 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.223 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.233 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.245 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.257 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.269 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.280 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.295 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.305 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.316 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.328 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.342 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.353 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.364 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.382 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.393 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.404 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.416 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.428 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.442 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.452 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.464 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.476 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.490 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.500 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.514 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.526 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.538 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.549 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.562 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.572 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.584 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.597 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.608 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.620 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.633 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.643 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.655 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.680 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.692 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.703 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.717 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.729 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.740 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.754 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.765 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.776 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.790 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.801 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.813 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.824 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.835 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.849 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.860 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.871 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.885 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.896 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.907 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.920 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.931 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.942 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.956 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.175.967 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.175.990 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.002 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.016 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.028 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.040 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.053 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.065 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.076 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.090 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.101 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.114 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.126 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.139 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.150 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.180 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.192 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.203 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.217 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.229 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.240 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.254 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.264 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.277 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.290 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.302 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.321 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.332 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.343 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.356 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.369 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.380 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.391 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.403 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.410 [stream.cc:237] 619419 FreeStreamId: Free stream_id=2. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.415 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.426 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.432 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.441 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=0, tsId=0, drvFlag=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.446 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.460 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.473 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.483 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.495 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.507 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.520 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.531 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.542 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.556 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.566 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.578 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.596 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.607 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.618 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.631 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.644 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.655 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.666 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.678 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.689 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.703 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.714 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.725 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.738 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.749 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.760 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.772 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.785 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.796 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.807 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.819 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.831 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.845 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.855 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.867 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.880 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.896 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.907 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.918 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.930 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.944 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.954 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.951 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=0, tsId=0, drvFlag=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.965 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.176.990 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.176.994 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.018 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.015 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=2, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.030 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.048 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.049 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=2, device_id=0, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.060 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.078 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.091 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.102 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.113 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.124 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.136 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.150 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.160 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.178 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.190 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.201 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.213 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.227 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.237 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.248 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.260 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.272 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.283 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.297 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.307 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.319 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.333 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.344 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.355 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.368 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.380 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.393 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.403 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.415 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.426 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.438 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.449 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.463 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.491 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.503 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.515 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.529 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.540 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.553 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.566 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.577 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.588 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.602 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.613 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.625 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.638 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.650 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.662 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.676 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.686 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.699 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.713 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.724 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.736 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.749 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.760 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.772 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.785 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.807 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.818 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.830 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.843 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.857 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.869 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.881 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.894 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.905 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.917 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.931 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.942 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.953 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.177.965 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.976 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.177.990 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.001 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.013 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.027 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.037 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.049 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.062 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.074 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.086 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.099 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.118 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.128 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.140 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.153 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.163 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.175 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.186 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.198 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.211 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.221 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.232 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.243 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.254 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.266 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.278 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.291 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.301 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.313 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.324 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.335 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.348 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.359 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.370 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.383 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.404 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.416 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.427 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.440 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.453 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.464 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.476 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.490 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.501 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.513 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.527 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.538 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.548 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.560 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.572 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.585 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.596 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.609 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.622 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.633 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.645 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.659 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.670 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.681 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.695 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.719 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.729 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.741 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.754 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.765 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.776 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.788 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.800 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.813 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.823 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.835 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.846 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.860 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.870 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.882 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.893 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.905 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.918 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.929 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.940 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.952 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.964 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.977 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.178.987 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.178.999 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.017 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.027 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.039 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.050 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.062 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.075 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.085 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.097 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.109 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.122 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.132 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.144 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.157 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.168 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.180 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.192 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.206 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.217 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.229 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.243 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.253 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.265 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.276 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.290 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.300 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.320 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.334 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.346 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.357 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.369 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.380 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.394 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.405 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.416 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.428 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.442 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.452 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.463 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.477 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.488 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.499 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.511 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.522 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.535 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.545 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.557 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.568 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.580 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.592 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.610 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.620 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.632 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.643 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.655 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.669 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.679 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.690 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.702 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.715 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.726 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.737 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.750 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.762 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.774 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.785 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.797 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.810 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.820 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.832 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.843 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.857 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.867 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.879 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.890 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.913 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.924 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.935 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.946 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.958 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.972 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.179.982 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.179.993 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.005 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.017 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.030 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.040 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.051 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.065 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.075 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.087 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.098 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.110 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.123 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.133 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.144 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.164 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.177 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.193 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.206 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.226 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.238 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.251 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.262 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.274 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.285 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.297 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.310 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.321 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.332 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.343 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.357 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.369 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.380 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.394 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.405 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.417 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.428 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.440 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.452 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.466 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.475 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.487 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.498 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.511 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.528 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.538 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.549 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.561 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.572 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.586 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.596 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.608 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.619 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.633 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.644 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.656 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.669 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.680 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.691 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.703 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.714 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.728 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.738 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.750 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.761 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.773 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.784 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.798 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.808 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.824 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.836 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.848 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.859 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.872 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.883 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.895 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.906 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.918 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.930 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.944 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.953 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.965 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.180.978 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.180.989 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.000 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.012 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.025 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.036 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.048 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.059 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.071 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.085 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.095 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.116 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.127 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.139 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.150 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.164 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.174 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.186 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.197 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.209 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.220 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.234 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.244 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.256 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.269 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.280 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.291 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.303 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.316 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.330 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.340 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.351 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.363 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.377 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.387 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.398 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.423 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.435 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.446 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.458 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.471 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.483 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.496 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.510 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.521 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.534 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.547 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.558 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.568 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.580 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.591 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.605 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.615 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.627 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.640 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.651 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.662 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.676 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.687 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.700 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.712 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.735 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.747 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.761 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.772 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.783 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.797 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.808 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.820 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.834 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.845 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.856 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.866 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.879 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.893 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.904 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.916 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.929 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.940 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.952 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.181.965 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.976 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.181.987 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.001 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.012 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.024 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.041 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.052 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.064 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.076 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.088 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.102 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.112 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.124 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.136 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.149 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.159 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.171 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.184 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.195 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.206 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.219 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.232 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.243 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.254 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.268 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.278 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.289 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.301 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.315 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.325 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.356 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.367 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.382 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.395 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.407 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.418 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.432 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.443 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.454 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.468 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.479 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.491 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.502 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.513 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.525 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.538 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.549 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.560 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.575 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.586 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.598 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.612 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.624 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.638 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.653 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.664 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.676 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.687 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.699 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.711 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.725 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.736 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.748 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.761 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.772 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.783 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.794 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.806 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.818 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.829 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.843 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.853 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.865 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.877 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.890 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.901 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.913 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.925 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.938 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.957 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.970 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.182.981 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.182.993 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.007 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.017 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.028 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.040 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.052 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.064 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.077 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.088 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.100 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.114 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.124 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.135 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.147 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.159 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.172 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.182 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.193 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.205 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.217 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.229 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.242 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.258 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.269 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.281 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.292 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.306 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.316 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.327 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.339 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.352 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.362 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.374 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.387 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.401 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.411 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.422 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.434 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.446 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.457 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.471 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.481 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.492 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.503 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.516 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.527 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.539 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.555 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.567 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.578 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.592 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.603 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.614 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.628 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.639 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.650 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.663 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.674 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.685 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.696 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.708 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.721 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.732 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.743 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.754 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.768 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.779 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.790 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.801 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.813 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.826 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.839 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.853 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.865 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.876 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.888 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.899 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.911 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.925 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.935 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.946 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.957 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.969 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.183.980 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.183.992 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.004 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.018 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.027 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.039 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.051 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.064 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.078 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.089 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.100 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.112 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.124 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.141 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.151 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.173 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.188 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.199 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.213 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.224 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.235 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.247 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.258 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.270 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.281 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.292 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.304 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.303 [stream.cpp:87]619419 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.319 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.340 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.354 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.365 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.377 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.390 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.401 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.184.392 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaac2b2c890. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.414 [callback.cpp:116]619610 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.432 [subscribe.cc:308] 619610 GetGroupIdByThreadId: can not find subscribe info, threadId=281471259520928 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.430 [stream.cpp:75]619419 aclrtDestroyStream: start to execute aclrtDestroyStream [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.454 [api_impl.cc:5043] 619610 ProcessReport: get groupId fail, threadIdentifier=281471259520928, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.473 [callback.cpp:130]619610 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] GE(619419,python3.9):2025-04-03-09:35:52.184.474 [stream_manage_func_registry.cc:22]619419 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.497 [stream.cpp:80]619419 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.518 [api_impl.cc:1228] 619419 StreamDestroy: stream_id=3. [INFO] DEVICE(619419,ffff22702fa0,python3.9):2025-04-03-09:35:52.184.534 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.543 [acl_resource_manager.cpp:814]619419 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.184.570 [single_op_manager.cc:50]619419 ReleaseResource:ReleaseResource in. resource id = 0xaaaac2c5d1a0 [INFO] GE(619419,python3.9):2025-04-03-09:35:52.184.595 [caching_allocator.cc:402]619419 TryFreeBlocks:Try free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:52.184.610 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:52.184.623 [npu_memory_allocator.cc:72]619419 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.636 [stream_executor.cpp:197]619419 RemoveExecutor: To remove executor by stream = 187650388906400 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.653 [acl_resource_manager.cpp:823]619419 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.670 [stream.cc:852] 619419 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.687 [stream.cc:938] 619419 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.706 [event.cc:1036] 619419 WaitForBusy: device_id=0, event_id=1 finished map already empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.725 [npu_driver.cc:626] 619419 EventIdFree: event id free begin, device_id=0, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.751 [raw_device.cc:1158] 619419 FreeEventIdFromDrv: device_id=0, event_id=1 tear down. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.766 [event.cc:61] 619419 ~Event: device_id=0, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.786 [npu_driver.cc:3803] 619419 SqCqFree: deviceId=0, tsId=0, sqId=136, cqId=136 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.822 [subscribe.cc:244] 619419 Delete: delete callback subscribe: groupId=65, threadId=281471251066784, devId=0, tsId=0, streamId=3, sqId=136, cqId=136 [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.184.824 [trs_sqcq.c:1491][ascend][curpid:619419,619611][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=0; type=1; ret=16) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.842 [stream.cc:1507] 619419 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.852 [npu_driver.cc:3833] 619611 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=0, tsId=0, groupId=65, drvRetCode=16. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.870 [api_impl.cc:5058] 619611 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.888 [stream_sqcq_manage.cc:451] 619419 FreeLogicCq: Return(0), threadIdentifier(281473133236256), devId(0), tsId(0), cqId(498), isFastCq(0). [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.902 [callback.cpp:132]619611 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.918 [npu_driver.cc:6163] 619419 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=3, logicCq=498. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.920 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.947 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.962 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.184.964 [npu_driver.cc:1180] 619419 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=498, drvFlag=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.976 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.184.994 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.009 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.019 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.033 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.044 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.056 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.068 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.081 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.092 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.104 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.117 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.129 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.140 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.155 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.165 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.177 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.190 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.214 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.226 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.238 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.252 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.264 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.278 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.290 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.303 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.316 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.326 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.338 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.349 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.361 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.372 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.385 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.396 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.408 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.421 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.432 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.444 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.457 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.468 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.482 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.496 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.516 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.526 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.538 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.550 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.563 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.573 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.585 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.599 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.610 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.621 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.635 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.645 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.656 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.670 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.680 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.691 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.706 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.716 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.728 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.740 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.754 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.764 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.776 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.789 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.800 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.823 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.835 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.846 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.860 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.871 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.884 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.897 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.911 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.921 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.933 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.946 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.958 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.185.971 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.983 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.185.994 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.009 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.020 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.032 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.045 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.056 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.067 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.081 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.092 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.104 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.117 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.149 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.161 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.178 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.189 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.203 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.214 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.227 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.240 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.251 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.264 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.279 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.290 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.304 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.315 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.327 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.339 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.353 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.364 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.376 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.389 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.399 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.411 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.425 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.435 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.448 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.467 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.478 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.490 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.501 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.513 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.517 [stream.cc:237] 619419 FreeStreamId: Free stream_id=3. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.527 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.541 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.546 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.558 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=0, tsId=0, drvFlag=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.565 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.585 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.596 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.608 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.622 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.632 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.644 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.656 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.669 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.679 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.691 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.703 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.716 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.727 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.738 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.760 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.772 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.786 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.798 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.809 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.823 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.834 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.846 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.859 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.870 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.882 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.893 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.906 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.919 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.930 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.942 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.955 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.966 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.186.977 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.186.991 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.001 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.013 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.026 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.024 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=0, tsId=0, drvFlag=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.037 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.048 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.066 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.078 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=3, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.085 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.099 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=3, device_id=0, tsId=0, streamFlag=1536 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.103 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.124 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.136 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.149 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.159 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.171 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.185 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.195 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.206 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.218 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.229 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.243 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.252 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.264 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.276 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.287 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.300 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.312 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.323 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.349 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.361 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.373 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.386 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.397 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.409 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.422 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.433 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.445 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.458 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.470 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.481 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.493 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.504 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.518 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.528 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.540 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.553 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.565 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.576 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.590 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.601 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.614 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.627 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.638 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.658 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.670 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.681 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.695 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.706 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.718 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.730 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.743 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.754 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.766 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.779 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.791 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.802 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.814 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.825 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.839 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.850 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.863 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.876 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.887 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.898 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.912 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.923 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.937 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.948 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.963 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.187.976 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.987 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.187.999 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.013 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.023 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.036 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.050 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.060 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.072 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.085 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.095 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.109 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.120 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.132 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.145 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.169 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.183 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.199 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.214 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.225 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.236 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.250 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.261 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.287 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.298 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.311 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.323 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.336 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.347 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.360 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.373 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.385 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.397 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.410 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.421 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.434 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.445 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.457 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.470 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.482 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.493 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.507 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.518 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.530 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.538 [event.cc:61] 619419 ~Event: device_id=0, event deconstruct, recordSize=0, waitSize=0, idSize=0 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.543 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.575 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.584 [stream.cpp:87]619419 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.587 [callback.cpp:116]619611 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.628 [subscribe.cc:308] 619611 GetGroupIdByThreadId: can not find subscribe info, threadId=281471251066784 [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.639 [api_impl.cc:5043] 619611 ProcessReport: get groupId fail, threadIdentifier=281471251066784, reCode=0x7160004 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.641 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaac2c5d1a0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.652 [callback.cpp:130]619611 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.671 [device.cpp:183]619419 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.698 [api_impl.cc:3419] 619419 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.715 [runtime.cc:3444] 619419 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] DEVICE(619419,ffff21ef2fa0,python3.9):2025-04-03-09:35:52.188.723 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.188.731 [api_impl.cc:3449] 619419 DeviceReset: Succ, userDeviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.754 [device.cpp:189]619419 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 0. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.783 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.848 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.877 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.897 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.915 [callback.cpp:37]619419 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.188.937 [callback.cpp:44]619419 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.964 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.188.991 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.396.842 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.396.869 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.396.906 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:52.396.920 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.396.940 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.397.015 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.181 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.182 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.184 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.190 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.259 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:52.420.262 [queue_client_comm.c:114][ascend][curpid:619419,619735][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.287 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.305 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.312 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.321 [tensor_data_transfer.cpp:1054]619741 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_image_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.327 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.333 [tensor_data_transfer.cpp:1056]619741 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.341 [tensor_data_transfer.cpp:1054]619745 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.348 [api_impl.cc:6121] 619741 MemQueuePeek: Start to peek queue on devId 0, qid is 3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.357 [tensor_data_transfer.cpp:1056]619745 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.361 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.362 [npu_driver.cc:4330] 619741 MemQueuePeek: peek mem queue, deviceId=0, qid=3, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.371 [api_impl.cc:6121] 619745 MemQueuePeek: Start to peek queue on devId 0, qid is 5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.377 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.387 [npu_driver.cc:4330] 619745 MemQueuePeek: peek mem queue, deviceId=0, qid=5, timeout=800. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.392 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.402 [tensor_data_transfer.cpp:1054]619743 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_scalar_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.414 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.419 [tensor_data_transfer.cpp:1056]619743 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.425 [tensor_data_transfer.cpp:1054]619739 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.427 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.430 [api_impl.cc:6121] 619743 MemQueuePeek: Start to peek queue on devId 0, qid is 4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.439 [tensor_data_transfer.cpp:1056]619739 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.447 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.452 [npu_driver.cc:4330] 619743 MemQueuePeek: peek mem queue, deviceId=0, qid=4, timeout=800. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.456 [api_impl.cc:6121] 619739 MemQueuePeek: Start to peek queue on devId 0, qid is 2. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.460 [npu_driver.cc:4338] 619735 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.460 [tensor_data_transfer.cpp:1054]619737 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.466 [npu_driver.cc:4330] 619739 MemQueuePeek: peek mem queue, deviceId=0, qid=2, timeout=800. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.476 [tensor_data_transfer.cpp:612]619735 acltdtReceiveTensorV2: queue is empty, device is 0, name is _npu_log [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.479 [tensor_data_transfer.cpp:1056]619737 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.498 [api_impl.cc:6121] 619737 MemQueuePeek: Start to peek queue on devId 0, qid is 1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.508 [npu_driver.cc:4330] 619737 MemQueuePeek: peek mem queue, deviceId=0, qid=1, timeout=800. [INFO] DEVICE(619419,fffef2798fa0,python3.9):2025-04-03-09:35:52.420.645 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.420.826 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is _npu_log [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.851 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:52.420.865 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:52.421.286 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is _npu_log [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.421.337 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:52.421.365 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619419,python3.9):2025-04-03-09:35:53.252.169 [queue_client_comm.c:114][ascend][curpid:619419,619741][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:53.252.169 [queue_client_comm.c:114][ascend][curpid:619419,619737][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:53.252.170 [queue_client_comm.c:114][ascend][curpid:619419,619745][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:53.252.178 [queue_client_comm.c:114][ascend][curpid:619419,619739][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619419,python3.9):2025-04-03-09:35:53.252.177 [queue_client_comm.c:114][ascend][curpid:619419,619743][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.263 [npu_driver.cc:4338] 619741 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.279 [tensor_data_transfer.cpp:612]619741 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_image_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.286 [npu_driver.cc:4338] 619737 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.301 [tensor_data_transfer.cpp:612]619737 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_dump [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.331 [npu_driver.cc:4338] 619745 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.347 [tensor_data_transfer.cpp:612]619745 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_histogram_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.362 [npu_driver.cc:4338] 619739 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.376 [tensor_data_transfer.cpp:612]619739 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_tensor_summary [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.394 [npu_driver.cc:4338] 619743 MemQueuePeek: queue empty [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.411 [tensor_data_transfer.cpp:612]619743 acltdtReceiveTensorV2: queue is empty, device is 0, name is ms_scalar_summary [INFO] DEVICE(619419,fffed2fdefa0,python3.9):2025-04-03-09:35:53.252.467 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619419,fffee9397fa0,python3.9):2025-04-03-09:35:53.252.508 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619419,fffed0f9efa0,python3.9):2025-04-03-09:35:53.252.516 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619419,fffed3ffefa0,python3.9):2025-04-03-09:35:53.252.523 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619419,fffed1fbefa0,python3.9):2025-04-03-09:35:53.252.537 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.252.795 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is ms_histogram_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.827 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.252.856 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=5. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.253.235 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is ms_histogram_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.253.289 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.253.320 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.253.367 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is ms_scalar_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.253.387 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 4. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.253.404 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=4. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.253.740 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is ms_scalar_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.253.770 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.253.792 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.253.851 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is ms_image_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.253.870 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 3. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.253.881 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=3. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.254.320 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is ms_image_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.254.349 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.254.370 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.254.413 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is ms_tensor_summary [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.254.431 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.254.444 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=2. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.254.796 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is ms_tensor_summary [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.254.823 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.254.843 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.254.896 [tensor_data_transfer.cpp:973]619419 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 0, name is ms_tensor_dump [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.254.914 [api_impl.cc:6065] 619419 MemQueueDestroy: Start to destroy queue on devId 0, qid is 1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.254.928 [npu_driver.cc:3989] 619419 MemQueueDestroy: deviceId=0, qid=1. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.255.320 [tensor_data_transfer.cpp:977]619419 acltdtDestroyChannel: acltdtDestroyChannel success, device is 0, name is ms_tensor_dump [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.255.343 [device.cpp:183]619419 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.255.363 [api_impl.cc:3419] 619419 DeviceReset: reset device, device_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.255.379 [runtime.cc:3444] 619419 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.255.395 [api_impl.cc:3449] 619419 DeviceReset: Succ, userDeviceId=0. [INFO] ASCENDCL(619419,python3.9):2025-04-03-09:35:53.255.406 [device.cpp:189]619419 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 0. [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.431 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.462 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.505 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.522 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.545 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_0 [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.562 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.589 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.605 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.255.629 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.794 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.828 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.847 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.866 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.895 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.922 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.957 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] PS(619419,ffff22fdefa0,python3.9):2025-04-03-09:35:53.255.971 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] DISTRIBUTED(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.255.975 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] PS(619419,ffff237eefa0,python3.9):2025-04-03-09:35:53.255.982 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] DISTRIBUTED(619419,fffee8b87fa0,python3.9):2025-04-03-09:35:53.256.014 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(619419,fffeb57aefa0,python3.9):2025-04-03-09:35:53.256.068 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.138 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.172 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.194 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.216 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.232 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.248 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.264 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.375 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.402 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.447 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.466 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.482 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.504 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.520 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.588 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.604 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.622 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.649 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.689 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.256.705 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.037 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.058 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.092 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.110 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.126 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.143 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.257.387 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:53.312.824 [awatchdog_monitor.c:160](tid:619459) thread 619751 not exist [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.608.796 [runtime.cc:5239] 619608 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:53.608.877 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:53.608.898 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619419,python3.9):2025-04-03-09:35:53.608.912 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619608] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.608.943 [runtime.cc:3077] 619608 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=0, devId=0, tdt status=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:53.609.086 [npu_driver.cc:551] 619608 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.656.774 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619419,ffff921ed020,python3.9):2025-04-03-09:35:53.656.920 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.305 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.506 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.538 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.862 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.882 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.963 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.002.980 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.476 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.499 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.513 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.528 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.549 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.003.564 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.207 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.235 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.346 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.363 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.898 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.920 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.934 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.947 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.962 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.975 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.004.989 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.001 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.013 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.028 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.038 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.050 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.065 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.076 [heartbeat.cc:145] [619419][DeInit]heartbeat deinit begin [INFO] HCCL(619419,python3.9):2025-04-03-09:35:54.005.090 [heartbeat.cc:183] [619419][DeInit]heartbeat deinit end [INFO] DEVICE(619419,ffff921ed020,python3.9):2025-04-03-09:35:54.005.419 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 0, group name: hccl_world_group [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:54.005.487 [hash_data.cpp:63] >>> (tid:619419) HashData uninitialize success [INFO] PROFILING(619419,python3.9):2025-04-03-09:35:54.005.515 [msprof_reporter_mgr.cpp:270] >>> (tid:619419) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.008.034 [api_impl.cc:2003] 619419 DevFree: device free mem=0x12c041200000 [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:54.010.203 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619419,ffff921ed020,python3.9):2025-04-03-09:35:54.010.242 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:54.010.399 [adx_dump_record.cpp:229][tid:619419] start to dump uninit [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:54.010.433 [adx_dump_record.cpp:239][tid:619419] dump uninit success [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:54.010.449 [adx_dump_record.cpp:674][tid:619733] exit record file thread [INFO] GE(619419,python3.9):2025-04-03-09:35:54.075.800 [operator_factory_impl.cc:392]619419 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.095 [operator_factory_impl.cc:392]619419 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.168 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.181 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.197 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.208 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.219 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.231 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.243 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.091.257 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.180.653 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.180.875 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] GE(619419,python3.9):2025-04-03-09:35:54.180.926 [opsproto_manager.cc:66]619419 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.237.862 [api_impl.cc:7151] 619419 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.238.115 [api_impl.cc:7135] 619419 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.239.931 [api_impl.cc:7151] 619419 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.240.088 [api_impl.cc:7135] 619419 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.242.221 [api_impl.cc:7121] 619419 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.245.925 [api_impl.cc:7121] 619419 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.253.443 [osal.cc:321] 619419 ThreadExit: ThreadExit start. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.828.796 [stars_engine.cc:1833] 619608 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:54.828.842 [stars_engine.cc:1836] 619608 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619419,ffff921ed020,python3.9):2025-04-03-09:35:54.829.082 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619419,python3.9):2025-04-03-09:35:54.865.116 [ops_kernel_builder_registry.cc:71]619419 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619419,python3.9):2025-04-03-09:35:54.865.217 [ops_kernel_builder_registry.cc:41]619419 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:56.192.198 [adx_server_manager.cpp:236][tid:619734] server manager stop [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:56.376.533 [adx_hdc_epoll.cpp:57][tid:619419] start to close hdc epoll [INFO] GE(619419,python3.9):2025-04-03-09:35:56.458.350 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac778f730]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.463.589 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac7793d60]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.466.388 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac77be1e0]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.469.861 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac7790d70]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.470.807 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac8bca360]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.471.735 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac8bf2350]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.473.035 [op_impl_registry_holder_manager.cc:31]619419 CloseHandle:start close handle, handle[0xaaaac8cb59d0]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.490.394 [mem_manager.cc:118]619419 Finalize:MemManager Finalize. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.490.439 [session_scope_mem_allocator.cc:73]619419 FreeAllMemory:Free all memory [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.870 [session_scope_mem_allocator.cc:73]619419 FreeAllMemory:Free all memory [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.904 [caching_allocator.cc:474]619419 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.941 [caching_allocator.cc:391]619419 FreeBlocks:Free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.958 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.970 [caching_allocator.cc:422]619419 FreeBlockBins:Free block bins. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.983 [caching_allocator.cc:474]619419 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.492.999 [caching_allocator.cc:391]619419 FreeBlocks:Free blocks. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.493.010 [caching_allocator.cc:358]619419 FreeCachedBlocks:Free cached blocks [INFO] GE(619419,python3.9):2025-04-03-09:35:56.493.021 [caching_allocator.cc:422]619419 FreeBlockBins:Free block bins. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.493.048 [mem_manager.cc:145]619419 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.493.067 [rts_caching_mem_allocator.cc:150]619419 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619419,python3.9):2025-04-03-09:35:56.493.125 [rts_caching_mem_allocator.cc:136]619419 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.146 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.268 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.288 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.300 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.312 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.326 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.357 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.370 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.382 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.396 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.407 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.419 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.430 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.478 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.489 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] HCCL(619419,python3.9):2025-04-03-09:35:56.596.501 [network_manager.cc:875] [619419]destroy all nic/vnic. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.603.543 [runtime.cc:1570] 619419 ~Runtime: deconstruct runtime [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.603.583 [runtime.cc:1577] 619419 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.030 [raw_device.cc:262] 619419 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.071 [engine.cc:188] 619419 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.099 [ringbuffer_maintain_task.cc:83] 619419 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.122 [stream.cc:1400] 619419 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.140 [stream.cc:2595] 619419 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.179 [npu_driver.cc:751] 619419 SqTaskSend: device_id=0, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.202 [stars_engine.cc:893] 619419 SendTask: Sq task send finished, device_id=0, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.217 [stream.cc:1494] 619419 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.234 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.247 [stream.cc:2886] 619419 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.263 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.277 [stream.cc:2886] 619419 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.293 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.309 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.327 [task.cc:185] 619419 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.345 [logger.cc:152] 619419 TaskFinished: device_id=0, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.369 [stars_engine.cc:437] 619419 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.385 [stream.cc:2886] 619419 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.404 [stars_engine.cc:524] 619419 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.419 [npu_driver.cc:5547] 619419 GetSqHead: deviceId=0, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.443 [stream.cc:852] 619419 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.463 [stream.cc:938] 619419 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.604.496 [stream.cc:1507] 619419 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.077 [stream_sqcq_manage.cc:451] 619419 FreeLogicCq: Return(0), threadIdentifier(281473133236256), devId(0), tsId(0), cqId(496), isFastCq(0). [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.126 [npu_driver.cc:6163] 619419 StreamUnBindLogicCq: success: deviceId=0, tsId=0, streamId=0, logicCq=496. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.209 [npu_driver.cc:1180] 619419 LogicCqFree: Succ:device_id=0,ts_id=0,type=2,logic_cq_id=496, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.231 [stream.cc:237] 619419 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.251 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.267 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.921 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.942 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.961 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.608.982 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=0, device_id=0, tsId=0, streamFlag=192 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.054 [stars_engine.cc:206] 619609 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.212 [stars_engine.cc:191] 619419 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.230 [osal.cc:225] 619419 Join: mmJoinTask failed, thread=281471260209056 does not exist! [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.243 [stars_engine.cc:150] 619419 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.270 [runtime.cc:5239] 619419 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=0, deviceId=0. [INFO] TDT(619419,python3.9):2025-04-03-09:35:56.609.291 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619419] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.308 [runtime.cc:3003] 619419 StopAicpuExecutor: TsdClose success, userDeviceId=0, devId=0, tdt status=0 [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.609.326 [atrace_api.c:118](tid:619419) AtraceDestroy start. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.609.372 [tracer_mgr_operate.c:248](tid:619419) destroy object RUNTIME_ATRACE_DEV0_TS0, exitSave(false). [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.609.393 [atrace_api.c:120](tid:619419) AtraceDestroy end. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.414 [stream.cc:237] 619419 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.430 [stream_sqcq_manage.cc:216] 619419 DeAllocStreamSqCq: deviceId=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.446 [npu_driver.cc:966] 619419 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.662 [npu_driver.cc:974] 619419 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=0, tsId=0, drvFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.680 [stream_sqcq_manage.cc:228] 619419 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.710 [npu_driver.cc:432] 619419 StreamIdFree: stream id free begin, device_id=0, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.609.725 [npu_driver.cc:441] 619419 StreamIdFree: Stream id free success, stream_id=1, device_id=0, tsId=0, streamFlag=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.620.051 [stars_engine.cc:58] 619419 ~StarsEngine: Destructor. [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.620.086 [engine.cc:84] 619419 ~Engine: Destructor. [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.620.976 [trs_dev_drv.c:370][ascend][curpid:619419,619419][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=0; fd=37; mmapFd=38) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.008 [devmm_svm.c:2238][ascend][curpid:619419,619419][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=0) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.027 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.048 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=10; free_cnt=10) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.067 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.087 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.105 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev0 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.123 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev0 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.143 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev0 Cached_size:1961984Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.159 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev0 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.175 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev0 Cached_size:2097152Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.191 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev0 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.208 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev0 Cached_size:2097152Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.224 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev0 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.257 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev0 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.276 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev0 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.295 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev0 Cached_size:23068672Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.309 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev0 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.329 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev0 Cached_size:6291456Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.346 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev0 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.364 [svm_mem_statistics.c:140][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev0 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.621.383 [svm_mem_statistics.c:146][ascend][curpid:619419,619419][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev0 Cached_size:16777216Bytes [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.693.503 [drv_log_user.c:623][ascend][curpid:619419,619419][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=0; phyid=0; vfid=0; hostpid=619419; devpid=319) [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.693.555 [npu_driver.cc:3045] 619419 DeviceClose: Close device success, device_id=0, tsId=0 [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.693.578 [raw_device.cc:125] 619419 ~Device: deconstruct device [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.802.750 [plugin_old_arch.cc:25] 619419 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.802.815 [runtime_keeper.cc:167] 619419 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619419,python3.9):2025-04-03-09:35:56.803.774 [task_fail_callback_data_manager.cc:63] 619419 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.917.667 [stacktrace_signal.c:188](tid:619419) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.991.192 [atrace_client_communication.c:141](tid:619419) send [end] msg. [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.993.619 [hdc_core.c:1982][ascend][curpid:619419,619419][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4278; pid=619419) [INFO] IDEDD(619419,python3.9):2025-04-03-09:35:56.993.670 [hdc_api.cpp:671][tid:619419] connect succ, peer_node: 0, peer_devid: 0 [INFO] DRV(619419,python3.9):2025-04-03-09:35:56.993.958 [hdc_client.c:417][ascend][curpid:619419,619419][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4278) [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:56.994.025 [atrace_client_communication.c:164](tid:619419) send [end] msg success. [INFO] DRV(619419,python3.9):2025-04-03-09:35:58.062.241 [hdc_client.c:417][ascend][curpid:619419,619603][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4252) [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:58.062.365 [atrace_client_core.c:127](tid:619603) atrace receive thread exited, devId=0. [INFO] ATRACE(619419,python3.9):2025-04-03-09:35:58.063.105 [awatchdog_monitor.c:234](tid:619419) AwdMonitorExit [INFO] DRV(619419,python3.9):2025-04-03-09:35:58.210.680 [uda_user.c:147][ascend][curpid:619419,619419][drv][devmng][uda_char_dev_open]The fd and pid info: (fd=3; pid=619419) [INFO] DRV(619419,python3.9):2025-04-03-09:35:58.210.837 [uda_user.c:317][ascend][curpid:619419,619419][drv][devmng][_uda_init]User info. (admin_flag=1; local_flag=0; max_dev_num=100; max_udev_num=1124; uda_dev_num_davinci=8; uda_dev_num_kunpeng=0) output_allreduce/worker_2.log0000644000175100017500000670446514773362776017143 0ustar jenkinsHwHiAiUser[INFO] DEBUG(619427,ffff8eb97020,python3.9):2025-04-03-09:35:26.798.046 [mindspore/ccsrc/debug/data_dump/data_dumper.cc:43] CpuDumpRegister] Register DataDumper for cpu backend. [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:26.816.286 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.301 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.395 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.450 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.473 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.494 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.514 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.537 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.556 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.575 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.594 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.610 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.627 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.644 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.661 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.678 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.698 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.714 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.731 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.749 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.763 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.776 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_UINT8,DT_INT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.817 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.834 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.851 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.869 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.883 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.916 [ir_data_type_symbol_store.cc:240]619427 DeclareSymbol:Bind symbol T1 with ordered list-dtypes [DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_INT4,DT_UINT8,DT_INT8,DT_INT16,DT_INT16,DT_INT32,DT_INT32,DT_INT32,DT_INT32,DT_INT64,DT_INT64,DT_FLOAT,DT_BFLOAT16,DT_BFLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.936 [ir_data_type_symbol_store.cc:240]619427 DeclareSymbol:Bind symbol T2 with ordered list-dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,DT_INT32,DT_INT16,DT_FLOAT,DT_INT32,DT_INT16,DT_INT8,DT_UINT8,DT_INT4,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT16,DT_FLOAT,DT_FLOAT,DT_INT64,DT_INT16,DT_FLOAT16,DT_INT32,DT_FLOAT,DT_BFLOAT16,DT_FLOAT,DT_INT32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.958 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.975 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.223.994 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.014 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.031 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.050 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.068 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.085 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.104 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.125 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.141 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.171 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.194 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.211 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.224 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.253 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.268 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.284 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.305 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.324 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.341 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.360 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.377 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.392 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.410 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.427 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.444 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.460 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.478 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.495 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.512 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.529 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.546 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.562 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.580 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.597 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.616 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.635 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.649 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.665 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.689 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.704 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.718 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.736 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.752 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.765 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.782 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.798 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.811 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.825 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.841 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.854 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.868 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.886 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.903 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.917 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.933 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.945 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.960 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.977 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.224.991 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.007 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.021 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.038 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.052 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.065 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.087 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.101 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.116 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.132 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.146 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.163 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.178 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.193 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.214 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.231 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.246 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.261 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.275 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.292 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.307 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.322 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.335 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.349 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.367 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.383 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.397 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.414 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.427 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.441 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.454 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.475 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.488 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.512 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.529 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.541 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.560 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.576 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.592 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.603 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x3 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.616 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.632 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.650 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.666 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.679 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.695 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.711 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.725 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.741 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.753 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.769 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.785 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.801 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.817 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.829 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.844 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.860 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.882 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.897 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.912 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.926 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.943 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.957 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.973 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.225.985 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.002 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.018 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.032 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.048 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.066 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.087 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Dynamic input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.103 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.121 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.139 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.157 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.170 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.183 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.199 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.215 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.230 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T1 for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.245 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T2 for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.271 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T2 for Required input x3 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.287 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T2 for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.301 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T1 with allowed dtypes [DT_UINT8,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.318 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T2 with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT32,DT_INT16,DT_INT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.334 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.350 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.364 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.381 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.399 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.415 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.432 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.450 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.463 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.486 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.515 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x1 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.532 [ir_data_type_symbol_store.cc:173]619427 SetInputSymbol:Create symbol T for Required input x2 [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.544 [ir_data_type_symbol_store.cc:219]619427 SetOutputSymbol:Create symbol expression T for Required output y [INFO] GE(619427,python3.9):2025-04-03-09:35:27.226.562 [ir_data_type_symbol_store.cc:182]619427 DeclareSymbol:Bind symbol T with allowed dtypes [DT_FLOAT,DT_FLOAT16,DT_INT8,DT_INT32,DT_UINT8,DT_INT16,DT_UINT16,DT_UINT32,DT_INT64,DT_UINT64,DT_DOUBLE,DT_BOOL,DT_STRING,DT_COMPLEX64,DT_COMPLEX128,DT_QINT8,DT_QINT16,DT_QINT32,DT_QUINT8,DT_QUINT16,DT_RESOURCE,DT_BFLOAT16,DT_COMPLEX32,] [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.536 [awatchdog_core.c:99](tid:619427) watchdog get platform host side [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.666 [awatchdog_monitor.c:127](tid:619427) create monitor thread successfully, tid :281472439033760 [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.688 [awatchdog_monitor.c:226](tid:619427) AwdMonitorInit [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.867 [trace_attr.c:99](tid:619427) doesn't set env ASCEND_LOG_DEVICE_FLUSH_TIMEOUT, use default timeout=0ms. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.896 [trace_attr.c:189](tid:619427) attr init success, timeout=0ms. [WARNING] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.911 [trace_system_api.c:115](tid:619427) env ASCEND_WORK_PATH is null. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.964 [trace_recorder.c:40](tid:619427) home_directory=/home/jenkins. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.983 [trace_recorder.c:121](tid:619427) use root path: /home/jenkins/ascend [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.233.996 [trace_event.c:151](tid:619427) init trace event success [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.234.049 [atrace_client_core.c:185](tid:619427) atrace client init success. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.265 [stacktrace_dumper.c:218](tid:619427) path = /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/bin/asc_dumper [WARNING] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.289 [trace_system_api.c:115](tid:619427) env ASCEND_COREDUMP_SIGNAL is null. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.369 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 2 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.389 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 15 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.406 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 3 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.421 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 4 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.436 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 5 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.450 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 6 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.463 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 7 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.479 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 8 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.494 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 11 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.510 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 24 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.523 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 25 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.537 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 31 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.553 [stacktrace_signal.c:165](tid:619427) register signal handler for signo 35 succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.568 [stacktrace_signal.c:168](tid:619427) trace signal init succeed. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:27.235.582 [trace_core.c:51](tid:619427) trace init success. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.236.304 [task_fail_callback_data_manager.cc:58] 619427 TaskFailCallBackManager: Constructor. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.246.503 [operator_factory_impl.cc:232]619427 RegisterInferShapeV2Func:operator infer shape v2 funcs init [INFO] GE(619427,python3.9):2025-04-03-09:35:27.246.537 [operator_factory_impl.cc:246]619427 RegisterInferShapeRangeFunc:operator infer shape range funcs init [INFO] GE(619427,python3.9):2025-04-03-09:35:27.246.551 [operator_factory_impl.cc:239]619427 RegisterInferDataTypeFunc:operator infer data type funcs init [INFO] HCCL(619427,python3.9):2025-04-03-09:35:27.254.004 [adapter_rts.cc:2720] [619427][adapter_rts.cc][CallBackInitRts] g_deviceType [6] g_deviceLogicId [-1] g_devicePhyId [-1] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.268.957 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Allreduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.269.029 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Reduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:27.269.061 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:27.279.292 [prof_cann_plugin.cpp:256] (tid:619427) Register module[6] callback. [INFO] OP(619427,python3.9):2025-04-03-09:35:27.329.550 [nnopbase_args_pool.cpp:123][NNOP][GetCacheSizeLimit][619427] cachelimit is 10000 [INFO] OP(619427,python3.9):2025-04-03-09:35:27.333.682 [op_dfx.cpp:230][NNOP][RecordOpArgCallback][619427] RecordOpArgCallback, get record op arg flag = 0 [INFO] OP(619427,python3.9):2025-04-03-09:35:27.333.731 [op_dfx.cpp:54][NNOP][SystemConfig][619427] enable debug kernel flag 0. [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:27.333.752 [prof_cann_plugin.cpp:256] (tid:619427) Register module[61] callback. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.389.100 [mem_manager.cc:138]619427 RegisterAllocatorManager:MemManager RegisterAllocatorManager. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.396.502 [cache_policy.cc:47]619427 Create:[CachePolicy] Create CachePolicy with match_policy: 0, aging_policy: 0 success; [INFO] GE(619427,python3.9):2025-04-03-09:35:27.411.711 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for AddN. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.411.857 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for BatchMatMulV2. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.411.967 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Add. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.411.996 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Mul. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.020 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Less. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.043 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Sub. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.066 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for RealDiv. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.088 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Equal. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.112 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for NotEqual. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.135 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Greater. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.165 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Maximum. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.192 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Minimum. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.217 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LogicalAnd. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.239 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LogicalOr. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.261 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Div. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.282 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LessEqual. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.306 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for SquaredDifference. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.328 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for GreaterEqual. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.350 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for DivNoNan. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.372 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LeakyReluGrad. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.472 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ClipByValue. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.587 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ConcatV2. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.614 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ConcatV2D. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.716 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Abs. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.743 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Relu. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.765 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for IsNan. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.788 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Sign. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.809 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Exp. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.830 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Cast. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.852 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LogicalNot. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.875 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Neg. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.897 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Sqrt. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.918 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Rsqrt. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.938 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Pow. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.962 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ZerosLike. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.412.984 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Sigmoid. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.005 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for BiasAdd. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.028 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Square. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.049 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Erf. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.072 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Reciprocal. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.094 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LeakyRelu. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.117 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Tanh. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.215 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ExpandDims. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.324 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Fill. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.434 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for GatherV2. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.542 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for LayerNorm. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.643 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for MatMul. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.670 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for MatMulV2. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.769 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Pack. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.885 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Pad. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.413.910 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for PadD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.005 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Range. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.116 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceSum. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.141 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMax. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.164 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMin. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.186 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceProd. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.209 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMean. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.230 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceAll. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.261 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceAny. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.283 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceSumD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.304 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMaxD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.328 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMinD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.350 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceProdD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.368 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceMeanD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.394 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceAllD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.415 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for ReduceAnyD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.510 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Reshape. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.611 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Select. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.722 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Slice. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.830 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Split. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.857 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for SplitV. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.878 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for SplitD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.900 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for SplitVD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.414.998 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Squeeze. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.111 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for StridedSliceD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.137 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for StridedSlice. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.233 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Tile. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.261 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for TileD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.357 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Transpose. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.478 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for TransposeD. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.580 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Unpack. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.415.678 [addn.cc:29]619427 InferSymbolShape:Register infer symbol shape for Unsqueeze. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.041 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type ConcatV2D [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.147 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type ExpandDims [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.258 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Fill [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.353 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Pack [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.454 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type ReduceProd [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.559 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Shape [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.651 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Squeeze [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.748 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type StridedSlice [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.765 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type StridedSliceD [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.855 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Tile [INFO] GE(619427,python3.9):2025-04-03-09:35:27.416.950 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Unpack [INFO] GE(619427,python3.9):2025-04-03-09:35:27.417.049 [symbolic_kernel_factory.cc:27]619427 RegisterCreator:Register host symbolic kernel for node type Unsqueeze [INFO] GE(619427,python3.9):2025-04-03-09:35:27.423.954 [process_node_engine_manager.cc:140]619427 RegisterEngine:Register ProcessNodeEngine id:PS success. [INFO] GE(619427,python3.9):2025-04-03-09:35:27.424.084 [process_node_engine_manager.cc:140]619427 RegisterEngine:Register ProcessNodeEngine id:NPU success. [INFO] DVPP(619427,python3.9):2025-04-03-09:35:27.430.770 [PlatformInfo.cpp:85][DVPP] [InitPara:85] [T2] chip type is 5. [INFO] DVPP(619427,python3.9):2025-04-03-09:35:27.430.801 [PlatformInfo.cpp:94][DVPP] [InitPara:94] [T2] run mode is 1. [INFO] DVPP(619427,python3.9):2025-04-03-09:35:27.431.316 [dvpp_dev_hdc.cpp:750][WRAP] [StartHostPacketCountTimer:750] [tid:619427] start packet count timer [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:27.453.135 [prof_atls_plugin.cpp:117] (tid:619427) RegisterProfileCallback, callback type is 7 [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:27.453.171 [prof_atls_plugin.cpp:117] (tid:619427) RegisterProfileCallback, callback type is 8 [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.533.396 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ascend.so.2 success. [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.533.877 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_ascend plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_ascend.so success. [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.587.164 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:377] LoadPlugin] Load libmindspore_ops_host plugin file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/libmindspore_ops_host.so success. [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.587.306 [mindspore/core/utils/ms_context.cc:105] SetDeviceId] Set MS_CTX_DEVICE_ID by env DEVICE_ID to: 2 [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.587.346 [mindspore/core/utils/ms_context.cc:165] set_backend_policy] ms set context backend policy:ge [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.588.126 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_base_symbol.cc:63] LoadAclBaseApiSymbol] Load acl base api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.589.804 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_compiler_symbol.cc:39] LoadAclOpCompilerApiSymbol] Load acl op compiler api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.589.886 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_mdl_symbol.cc:115] LoadAclMdlApiSymbol] Load acl mdl api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.589.925 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_op_symbol.cc:55] LoadAclOpApiSymbol] Load ascend op api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.590.020 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_prof_symbol.cc:49] LoadProfApiSymbol] Load acl prof api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.590.055 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_allocator_symbol.cc:48] LoadAclAllocatorApiSymbol] Load acl allocator api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.590.114 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_rt_symbol.cc:149] LoadAclRtApiSymbol] Load acl rt api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.590.142 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_symbol.cc:34] LoadAclApiSymbol] Load acl base api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.591.888 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/acl_tdt_symbol.cc:82] LoadAcltdtApiSymbol] Load acl tdt api success! [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.591.918 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:79] LoadAscendApiSymbols] Load ascend api success! [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.591.954 [acl.cpp:753]619427 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.592.017 [runtime_keeper.cc:78] 619427 GetDeviceType: hardwareVersion=0x503 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.595.936 [plugin_old_arch.cc:16] 619427 ConstructRuntimeImpl: RuntimeImpl construct success, rt = 0xaaaadab79e80 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.595.970 [runtime_keeper.cc:149] 619427 CreateRuntimeImpl: Open libruntime_v100.so success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.595.987 [runtime_keeper.cc:221] 619427 BootRuntime: new Runtime ok, Runtime_alloc_size 12600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.017 [runtime.cc:1387] 619427 CheckHaveDevice: hardwareVersion=0x503 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.048 [runtime.cc:1163] 619427 InitChipAndSocType: Runtime init: device type=5, soc type=18, arch type=0, havedevice=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.076 [npu_driver.cc:4062] 619427 CheckSqcqIsSupportDevMem: sqcq support dev mem [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.099 [runtime.cc:5096] 619427 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.115 [runtime.cc:1173] 619427 InitApiImpile: ApiImpl:Runtime_alloc_size 8 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.127 [runtime.cc:1183] 619427 InitLogger: Logger:Runtime_alloc_size 32 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.145 [runtime.cc:1193] 619427 InitApiProfiler: RProfiler:Runtime_alloc_size 176 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.172 [profiler.cc:84] 619427 Init: Init ok, Runtime_alloc_size 48 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.200 [runtime.cc:1209] 619427 InitApiErrorDecorator: ApiErrorDecorator:Runtime_alloc_size 16. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.226 [runtime.cc:1219] 619427 InitThreadGuard: Runtime init:new ThreadGuard ok, Runtime_alloc_size 104. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.238 [runtime.cc:1230] 619427 InitStreamObserver: EngineStreamObserver:Runtime_alloc_size 8. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.249 [runtime.cc:1242] 619427 InitAicpuStreamIdBitmap: Bitmap(1024):Runtime_alloc_size 72. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.262 [runtime.cc:1264] 619427 InitCbSubscribe: The thread group num is 1024 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.275 [runtime.cc:1270] 619427 InitCbSubscribe: Runtime init:new CbSubscribe ok, Runtime_alloc_size 304. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.390 [runtime.cc:109] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.792 [runtime.cc:121] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.873 [runtime.cc:1314] 619427 InitLabelAllocator: Runtime init:new LabelAllocator ok, Runtime_alloc_size 131152. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.596.891 [runtime.cc:1322] 619427 InitSetRuntimeVersion: set runtime api ver success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.014 [runtime.cc:1338] 619427 InitNpuCollectPath: NPU_COLLECT_PATH is not set! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.083 [utils.cc:105] 619427 GetDriverPath: driver path is [Driver_Install_Path_Param=/usr/local/Ascend]. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.137 [runtime.cc:1664] 619427 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.178 [runtime.cc:1688] 619427 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.201 [runtime.cc:1953] 619427 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=preload_stack_16KB. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.228 [runtime.cc:1477] 619427 FindAndRegisterDcacheLockOp: register lock op success [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:27.597.250 [prof_atls_plugin.cpp:210] (tid:619427) Module[7] register callback of ctrl handle. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.271 [heterogenous.cc:19] 619427 ReadHeterogenousModeFromConfigIni: can not read ASCEND_LATEST_INSTALL_PATH! isHeterogenous=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:27.597.303 [api_c_soc.cc:70] 619427 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.597.321 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.597.337 [acl.cpp:760]619427 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.597.372 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.636.353 [mindspore/ccsrc/pybind_api/utils/log_adapter_py.cc:27] PyExceptionInitializer] Set exception handler [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.670.123 [mindspore/ccsrc/pybind_api/init.cc:250] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.670.582 [mindspore/ccsrc/pybind_api/init.cc:316] pybind11_init__c_expression] Start JitExecutorPy... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.671.026 [mindspore/ccsrc/pybind_api/init.cc:381] pybind11_init__c_expression] Start ParallelContext... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.671.836 [mindspore/ccsrc/pybind_api/init.cc:515] pybind11_init__c_expression] Start CostModelContext... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.672.301 [mindspore/ccsrc/pybind_api/init.cc:617] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.674.053 [mindspore/ccsrc/plugin/res_manager/ascend/symbol_interface/symbol_utils.cc:65] LoadAscendApiSymbols] Ascend api is already loaded. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.674.098 [acl.cpp:753]619427 aclrtGetSocName: start to execute aclrtGetSocName. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.674.126 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:27.674.140 [acl.cpp:760]619427 aclrtGetSocName: execute aclrtGetSocName successfully [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:27.674.164 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:408] SetContextSocVersion] The soc version :Ascend910B3 [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.677.343 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P5." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.679.147 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9362." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.700.009 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_95." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.708.911 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9382." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.711.914 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9361." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.714.929 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_5591." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.743.134 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend910_9392." [WARNING] FE(619427,python3.9):2025-04-03-09:35:28.757.787 [platform_info.cpp:853]619427 AssemblePlatformInfoVector:"Can not find soc info enum by Ascend310P7." [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.726.414 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[conv2d_data_rm_build_pass] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.728.237 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_conv3d] of op type[Conv3D] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.728.373 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:59][decorator] Fusion pass[trans_shape_depthwise_conv2d] of stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.730.634 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_elemwise] of op type[None] op pattern[ElemWise] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.730.772 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_for_broadcast] of op type[None] op pattern[Broadcast] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.730.891 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_biasadd] of op type[BiasAdd] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.731.017 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_requant_s16] of op type[AscendRequantS16] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.731.124 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_of_bninference] of op type[BNInferenceD] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.732.278 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection] of op type[FullyConnection] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.732.417 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_fully_connection_compress] of op type[FullyConnectionCompress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.357 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.491 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmul_bias_shape] of op type[MatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.607 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_matmulcompress_bias_shape] of op type[MatMulV2Compress] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.718 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMulV2] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.823 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_batch_matmul_shape] of op type[BatchMatMul] op pattern[None] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [INFO] TBE(619427,python3.9):2025-04-03-09:35:29.734.931 [../../../../../../latest/python/site-packages/tbe/common/register/fusion_pass_mgr.py:114][decorator] Fusion pass[trans_shape_transdata_fixpipe] of op type[None] op pattern[BatchMatmul] stage[InvokeStage.STAGE_BUILD] priority[Priority.NORMAL] has been registered. [WARNING] TBE(619427,python3.9):2025-04-03-09:35:30.352.469 [../../../../../../latest/python/site-packages/te/platform/cce_policy.py:50][OpImplPolicy] te.platform.cce_policy.OpImplPolicy is deprecated, use te_fusion.fusion_util.OpImplPolicy instead. [INFO] TBE(619427,python3.9):2025-04-03-09:35:34.825.678 [../../../../../../latest/python/site-packages/tbe/common/rl_bank/dynamic_op_filter.py:373][_need_to_tune_all_op] Auto search dynamic filter info: tune_all_op is False. [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.087.810 [mindspore/core/utils/ms_context.cc:306] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.087.906 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.087.943 [mindspore/core/utils/ms_context.cc:310] SetDeviceTargetFromInner] Set memory_optimize_level to O0 as default on other device [INFO] GE(619427,python3.9):2025-04-03-09:35:35.133.313 [ops_kernel_builder_registry.cc:55]619427 OpsKernelBuilderRegistrar:Register kernel lib name = ops_kernel_info_hccl [INFO] GE(619427,python3.9):2025-04-03-09:35:35.133.426 [ops_kernel_builder_registry.cc:28]619427 Register:Register OpsKernelBuilder successfully, kernel lib name = ops_kernel_info_hccl [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.172.191 [mindspore/core/utils/ms_context.cc:839] UseNewBackend] Use the new backend. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.172.306 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:27] Init] Start init lazy fusion. [INFO] KERNEL(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.172.397 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_flags.cc:174] LazyFusionFlags] lazy_fusion_flags :{"disable_ops":[],"dump_as_text":false,"enable_ops_only":[],"flush_threshold":100,"online_tuning":false,"opt_level":0,"synchronize":false} [INFO] KERNEL(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.172.421 [mindspore/ccsrc/plugin/device/ascend/kernel/dvm/lazy_fusion_op.cc:1636] LazyFusionAscendInit] Skip init lazy fusion. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:35.172.441 [mindspore/ccsrc/runtime/pynative/lazy_fusion.cc:31] Init] End init lazy fusion. [WARNING] TBE(619427,python3.9):2025-04-03-09:35:36.057.505 [../../../../../../latest/python/site-packages/te/tik/__init__.py:51][] 'from te import tik' is deprecated, please replace it with 'from tbe import tik'. [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:37.889.618 [mindspore/profiler/common/registry.py:36] registered module: CpuProfiler with name: CPU [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:37.890.344 [mindspore/profiler/common/registry.py:36] registered module: GpuProfiler with name: GPU [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:37.902.134 [mindspore/profiler/common/registry.py:36] registered module: NpuProfiler with name: Ascend [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:37.998.935 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:466] GetDeviceContext] Device context of device Ascend is not created yet. [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:37.999.011 [mindspore/hal/device.py:158] Backend Ascend is not created yet. [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.335 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.392 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:46] NodeBase] Cluster topo timeout is 600 seconds. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.417 [mindspore/ccsrc/include/backend/distributed/cluster/topology/node_base.h:51] NodeBase] Node timeout after exception is 30 seconds. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.449 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.719 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471464316832 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.848 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471455862688 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.000.874 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:29] TCPClient] Tcp client receiving message timeout is 15 seconds. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.018 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471447408544 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.162 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:189] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471438954400 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.193 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:170] Register] Start connecting heartbeat client. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.215 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.447 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 20 source: 127.0.0.1:58742, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.001.499 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 1 [WARNING] DISTRIBUTED(619427,ffff2da31fa0,python3.9):2025-04-03-09:35:38.001.501 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58742 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.586 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.612 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.630 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:178] Register] Start connecting business client. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.648 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:441] Connect] Can not found link destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.746 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:485] Connect] Connection 21 source: 127.0.0.1:58774, destination: 127.0.0.1:8118 [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:38.501.767 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:494] Connect] Waiting for the state of the connection to 127.0.0.1:8118 to be connected...Retry number: 2 [WARNING] DISTRIBUTED(619427,ffff2ea51fa0,python3.9):2025-04-03-09:35:38.501.783 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:79] ConnectedEventHandler] Connection from 127.0.0.1:58774 to 127.0.0.1:8118 is successfully created. System errno: Success [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:39.001.845 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:505] Connect] Connected to destination: 127.0.0.1:8118 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:39.001.870 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:73] Connect] Connected to the tcp server 127.0.0.1:8118 successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:39.002.148 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:222] Register] The compute graph node: 2 has been registered successfully. [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:39.002.282 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(1/1200). [INFO] DISTRIBUTED(619427,ffff2ca11fa0,python3.9):2025-04-03-09:35:39.002.338 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:270] Heartbeat] Interval of heartbeat lower and upper are 3 and 5 [INFO] DISTRIBUTED(619427,ffff2ca11fa0,python3.9):2025-04-03-09:35:39.002.370 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:274] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:39.502.365 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(2/1200). [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.002.466 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:246] BuildCluster] Topology build timed out., retry(3/1200). [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.582 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:247] BuildCluster] [PROF]BuildCluster costs 2502.17 msec. [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.614 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:249] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.634 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:342] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.655 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:356] PostProcess] This node 2 rank id: 2 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.674 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:360] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.703 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:371] PostProcess] Assigned for this worker port range is 8630 to 8885 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.727 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:253] BuildCluster] [PROF]PostBuildCluster costs 0.091 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.767 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.794 [mindspore/ccsrc/distributed/init.cc:52] Initialize] [PROF]distributed_cluster_init costs 2502.51 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.819 [mindspore/ccsrc/distributed/collective/collective_manager.cc:175] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.972 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:48] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.502.992 [mindspore/ccsrc/distributed/collective/collective_manager.cc:577] InitHostCommlib] Start initializing communication library on host side... [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.503.023 [mindspore/ccsrc/ps/core/abstract_node.cc:1368] InitNodeNum] The worker num:8, the server num:0, the scheduler ip:127.0.0.1, the scheduler port:8118 [WARNING] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.503.068 [mindspore/ccsrc/ps/core/file_configuration.cc:24] Initialize] The file: is not exist. [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.503.087 [mindspore/ccsrc/ps/core/file_configuration.cc:27] Initialize] Create directory for file: success. [WARNING] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.503.105 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:33] Start] Failed to initialize the configuration for this mccl collective node. [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.504.887 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:170] Init] The max connection is:10000 [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.504.912 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:179] Init] Initialize TcpServer with port range 8630 to 8885 [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.505.194 [mindspore/ccsrc/ps/core/abstract_node.cc:1358] InitNodeInfo] The node role:WORKER is generate uuid is:4945e387-64e4-4708-b17a-fa3aae919aec, the ip:127.0.0.1, the port:8630 [INFO] PS(619427,ffff17ffefa0,python3.9):2025-04-03-09:35:40.505.216 [mindspore/ccsrc/ps/core/abstract_node.cc:1447] operator()] The worker node or server node start a tcp server! [INFO] PS(619427,ffff17ffefa0,python3.9):2025-04-03-09:35:40.505.239 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:217] Start] Start tcp server! [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.506.873 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:116] Init] SSL is disable. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.507.033 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:89] SynchronizeAddresses] The address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_2 has been registered successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.507.059 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:40.507.213 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:115] SynchronizeAddresses] Waiting for the address of rank MCCL_COLLECTIVE_RANK_MS_WORKER_0 to be registered, retry 19 times. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.507.895 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:49] Start] The cpu collective rank 2 has been started successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.507.966 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:184] node_num] Number of role MS_WORKER is 8 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.049 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:81] Initialize] Query retry count is 400 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.073 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:89] Initialize] Interval of retry allgather hostname lower and upper are 1 and 2 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.111 [mindspore/ccsrc/distributed/collective/collective_manager.cc:597] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 2, global rank size: 8 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.149 [mindspore/ccsrc/distributed/collective/collective_manager.cc:198] Initialize] [PROF]InitHostCommlib costs 3005.29 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.202 [mindspore/ccsrc/distributed/collective/collective_manager.cc:640] AssignLocalRank] Host name for rank 2 is devserver-c233-2 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.327 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:564] GetHostNames] Worker gets host names {"hostnames":["devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2","devserver-c233-2"]} [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.400 [mindspore/ccsrc/distributed/collective/collective_manager.cc:661] AssignLocalRank] Successfully get all nodes' hostname. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.426 [mindspore/ccsrc/distributed/collective/collective_manager.cc:678] AssignLocalRank] The local rank id assigned for this process is 2 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.446 [mindspore/ccsrc/distributed/collective/collective_manager.cc:679] AssignLocalRank] The env 'DEVICE_ID' assigned for this process is: 2 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.476 [mindspore/ccsrc/distributed/collective/collective_manager.cc:697] AssignLocalRank] The device_id of ms_context is set to local rank id [2]. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.500 [mindspore/ccsrc/distributed/collective/collective_manager.cc:206] Initialize] [PROF]AssignLocalRank costs 0.297 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.620 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:236] Initialize] Start initializing device context. [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.660 [mindspore/core/utils/ms_context.cc:463] PrintJitLevelAndExecMode] The jit_level is: O0, and enable kernelbykernel executor in the PYNATIVE mode. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.508.686 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:250] Initialize] The current overflow detection mode is INFNAN. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.508.735 [device.cpp:365]619427 aclrtSetDeviceSatMode: start to execute aclrtSetDeviceSatMode, mode is 1 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.508.892 [api_impl.cc:249] 619427 CurrentContext: current ctx is nullptr! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.508.913 [api_impl.cc:6795] 619427 SetDeviceSatMode: Set saturation mode to 1 for runtime process. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.508.929 [device.cpp:371]619427 aclrtSetDeviceSatMode: successfully execute aclrtSetDeviceSatMode, mode is 1 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.509.009 [callback.cpp:37]619427 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.509.059 [callback.cpp:44]619427 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.509.086 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:235] Init] Start inititializing kernel runtime(SetDevice & CreateDefaultStream & MemoryManagerInitialization etc). [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.509.114 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.509.155 [device.cpp:143]619427 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.509.267 [npu_driver.cc:124] 619427 NpuDriver: Run mode=1 [0:offline, 1:online], env type=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.509.304 [api_impl.cc:2999] 619427 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.509.346 [runtime.cc:3602] 619427 DeviceRetain: [dlog api] DlogReportStart exist. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.509.603 [hdc_core.c:1982][ascend][curpid:619427,619427][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4225; pid=619427) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.509.809 [runtime.cc:3609] 619427 DeviceRetain: [dlog api] AtraceReportStart exist. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.509.870 [atrace_client_communication.c:103](tid:619427) send [hello] msg. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.509.892 [atrace_client_communication.c:63](tid:619427) send msg by adx[begin], resultLen=1024 bytes. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.510.018 [hdc_core.c:1982][ascend][curpid:619427,619427][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4226; pid=619427) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:43.510.038 [hdc_api.cpp:671][tid:619427] connect succ, peer_node: 0, peer_devid: 2 [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.511.391 [hdc_client.c:417][ascend][curpid:619427,619427][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4226) [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.427 [atrace_client_communication.c:65](tid:619427) send msg by adx[finish], ret=0. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.444 [atrace_client_communication.c:97](tid:619427) msgType=1, magic=49959, version=1. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.459 [atrace_client_communication.c:135](tid:619427) send [hello] msg success. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.560 [atrace_client_core.c:160](tid:619427) start to receive device[2] trace log. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.511.628 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.511.648 [runtime.cc:2855] 619427 startAicpuExecutor: TsdOpen devId=2, device mode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.511.665 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.511.685 [hdc_core.c:1982][ascend][curpid:619427,619573][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4227; pid=619427) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:43.511.712 [hdc_api.cpp:671][tid:619573] connect succ, peer_node: 0, peer_devid: 2 [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.725 [client_manager.cpp:160][GetPlatformInfo][tid:619427] [TsdClient] mode[1] chipType[5] [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.511.768 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619427] Get env[ASCEND_RT_VISIBLE_DEVICES] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.786 [client_manager.cpp:582][GetVisibleDevices][tid:619427] [TsdClient] Get env ASCEND_RT_VISIBLE_DEVICES []. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.803 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.819 [client_manager.cpp:125][GetInstance][tid:619427] [ClientManager] GetInstance, deviceId[2], g_runningMode[0], begin saving instance [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.859 [client_manager.cpp:435][IsHostEnvironment][tid:619427] [TsdClient] logicDeviceId is [2], hostaicpunum[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.878 [client_manager.cpp:464][GetClientRunMode][tid:619427] runningMode:0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.894 [client_manager.cpp:127][GetInstance][tid:619427] [ClientManager] Current mode:1 [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.511.907 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619427] Get env[AICPU_PROFILING_MODE] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.920 [client_manager.cpp:347][GetProfilingMode][tid:619427] Get AICPU_PROFILING_MODE[] [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.950 [atrace_client_communication.c:177](tid:619573) send [start] msg. [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.511.954 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619427] Get env[ASCEND_MODULE_LOG_LEVEL] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.511.975 [process_mode_base.cpp:149][GetLogLevel][tid:619427] [TsdClient] get ASCEND_GLOBAL_LOG_LEVEL [1] ASCEND_GLOBAL_EVENT_ENABLE [0] ASCEND_MODULE_LOG_LEVEL [] [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.511.998 [atrace_client_communication.c:189](tid:619573) send [start] msg success. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.081 [process_mode_base.cpp:161][GetLogLevel][tid:619427] [TsdClient] Get Log Level[001] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.100 [process_mode_manager.cpp:71][OpenProcess][tid:619427] [ProcessModeManager] enter into open process deviceId[2] rankSize[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.118 [process_mode_manager.cpp:615][CheckNeedToOpen][tid:619427] [TsdClient] get open para startCp[1] startHccp[0] (0:false 1:true) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.226 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[.], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.312 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[scene.info], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.385 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[Ascend-aicpu_syskernels.tar.gz], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.502 [client_manager.cpp:264][CheckPackageExists][tid:619427] [TsdClient] find aicpu package[Ascend-aicpu_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.520 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[ascend_install.info], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.614 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[version.info], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.703 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[..], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.512.833 [client_manager.cpp:261][CheckPackageExists][tid:619427] [TsdClient] check file[Ascend-aicpu_extend_syskernels.tar.gz], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.212 [client_manager.cpp:268][CheckPackageExists][tid:619427] [TsdClient] find extend package[Ascend-aicpu_extend_syskernels.tar.gz] in path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], deviceId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.232 [client_manager.cpp:283][CheckPackageExists][tid:619427] [TsdClient][deviceId=2] set extend package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_extend_syskernels.tar.gz] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.249 [client_manager.cpp:286][CheckPackageExists][tid:619427] [TsdClient][deviceId=2] set aicpu package path[/usr/local/Ascend/ascend-toolkit/latest/opp/Ascend/aicpu/], name[Ascend-aicpu_syskernels.tar.gz] [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.513.318 [devdrv_hccl.c:103][ascend][curpid:619427,619427][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619427) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.336 [process_mode_manager.cpp:396][InitTsdClient][tid:619427] [TsdClient][deviceId=2] get process sign success, procPid[619427] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.352 [process_mode_manager.cpp:403][InitTsdClient][tid:619427] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.388 [hdc_common.cpp:67][InitMsgSize][tid:619427] HdcCommon::InitMsgSize Start [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.404 [hdc_common.cpp:80][InitMsgSize][tid:619427] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.423 [hdc_client.cpp:184][CreateHdcSession][tid:619427] HdcClient::CreateHdcSession Start [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.510 [hdc_client.cpp:208][CreateHdcSession][tid:619427] [HdcClient] deviceId: 2 connect session [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.513.543 [hdc_core.c:1982][ascend][curpid:619427,619427][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4228; pid=619427) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.561 [hdc_client.cpp:219][CreateHdcSession][tid:619427] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.577 [hdc_client.cpp:230][CreateHdcSession][tid:619427] [HdcClient] deviceId: 2 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.597 [version_verify.cpp:36][SetVersionInfo][tid:619427] VersionVerify: send client version to server [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.643 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.669 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.513.686 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:64, features:{get check code retry}} [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.513.739 [internal_api.cpp:101][GetScheduleEnv][tid:619427] Get env[DATAMASTER_RUN_MODE] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.073 [version_verify.cpp:68][PeerVersionCheck][tid:619427] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.096 [version_verify.cpp:89][ParseVersionInfo][tid:619427] VersionVerify: pass client version info success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.111 [hdc_client.cpp:275][CheckHdcConnection][tid:619427] Service[2] create hdc success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.129 [version_verify.cpp:122][SpecialFeatureCheck][tid:619427] VersionVerify: new type[35], supported [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.514.147 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619427] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.180 [process_mode_manager.cpp:627][GetDeviceCheckCodeOnce][tid:619427] [TsdClient][deviceId=2] [sessionId=1] wait package info respond [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.514.198 [hdc_client.cpp:154][TsdRecvData][tid:619427] recv timeout is:150000 ms [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.175 [process_mode_msg_parse.cpp:148][PackageInfoMsgProc][tid:619427] [PackageInfoMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.193 [process_mode_msg_parse.cpp:151][PackageInfoMsgProc][tid:619427] [PackageInfoMsgProc] [sessionID=1] deviceId[2], checkCode[56684994] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.207 [hdc_client.cpp:172][TsdRecvData][tid:619427] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.224 [version_verify.cpp:122][SpecialFeatureCheck][tid:619427] VersionVerify: new type[64], supported [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.242 [hdc_client.cpp:411][Destroy][tid:619427] enter HdcClient::Destroy() function [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.515.258 [hdc_client.c:417][ascend][curpid:619427,619427][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4228) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.286 [hdc_client.cpp:394][DestroyClient][tid:619427] begin drvHdcClientDestroy [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.303 [hdc_client.cpp:399][DestroyClient][tid:619427] end drvHdcClientDestroy [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.319 [hdc_client.cpp:377][ClearClientPtr][tid:619427] begin HdcClient::ClearClientPtr [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.331 [hdc_client.cpp:385][ClearClientPtr][tid:619427] end HdcClient::ClearClientPtr [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.344 [hdc_client.cpp:418][Destroy][tid:619427] end HdcClient::Destroy() function [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.380 [process_mode_manager.cpp:371][SendAICPUPackage][tid:619427] [TsdClient]the checksum of host package[56684994] is same to device[56684994], skip send package. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.400 [process_mode_manager.cpp:1870][SendExtendPackage][tid:619427] [TsdClient]the checksum of host extend package[9658754] is same to device[9658754], skip send package. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.515.420 [devdrv_hccl.c:103][ascend][curpid:619427,619427][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619427) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.437 [process_mode_manager.cpp:396][InitTsdClient][tid:619427] [TsdClient][deviceId=2] get process sign success, procPid[619427] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.450 [process_mode_manager.cpp:403][InitTsdClient][tid:619427] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.466 [hdc_common.cpp:67][InitMsgSize][tid:619427] HdcCommon::InitMsgSize Start [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.481 [hdc_common.cpp:80][InitMsgSize][tid:619427] msgMaxSize_ = 524224 bytes, msgShortHeadDataMaxSize_ = 524212 bytes, msgLongHeadDataMaxSize_ = 524200 bytes [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.507 [hdc_client.cpp:184][CreateHdcSession][tid:619427] HdcClient::CreateHdcSession Start [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.560 [hdc_client.cpp:208][CreateHdcSession][tid:619427] [HdcClient] deviceId: 2 connect session [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.515.591 [hdc_core.c:1982][ascend][curpid:619427,619427][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4229; pid=619427) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.608 [hdc_client.cpp:219][CreateHdcSession][tid:619427] [HdcClient] drvHdcSetSessionReference success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.621 [hdc_client.cpp:230][CreateHdcSession][tid:619427] [HdcClient] deviceId: 2 connect session and drvHdcSetSessionReference success, sessionId=1 [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.637 [version_verify.cpp:36][SetVersionInfo][tid:619427] VersionVerify: send client version to server [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.650 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:35, features:{check before send aicpu package}} [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.665 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:37, features:{check before send open qs message}} [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.515.678 [version_verify.cpp:52][SetVersionInfo][tid:619427] send feature_info:{msg_type:64, features:{get check code retry}} [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.004 [version_verify.cpp:68][PeerVersionCheck][tid:619427] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.023 [version_verify.cpp:89][ParseVersionInfo][tid:619427] VersionVerify: pass client version info success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.040 [hdc_client.cpp:275][CheckHdcConnection][tid:619427] Service[2] create hdc success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.056 [process_mode_manager.cpp:458][ConstructOpenMsg][tid:619427] [TsdClient] tsd get process sign successfully, procpid[619427] signSize[48] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.075 [version_verify.cpp:114][SpecialFeatureCheck][tid:619427] VersionVerify: previous type[6], supported [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.102 [process_mode_manager.cpp:88][OpenProcess][tid:619427] [ProcessModeManager] deviceId[2] sessionId[1] rankSize[0], wait sub process start respond [WARNING] TDT(619427,python3.9):2025-04-03-09:35:43.516.120 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619427] Get env[ASAN_RUN_MODE] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.516.132 [hdc_client.cpp:154][TsdRecvData][tid:619427] recv timeout is:150000 ms [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.726.743 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619427] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.726.770 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619427] [TsdClient] DeviceMsgProc recvMsg realDeviceId[2] msgType[8] localDevId[2] rspCode[0]heterogeneousSubPid[0], tsdSupportLevel_[63] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.726.789 [hdc_client.cpp:172][TsdRecvData][tid:619427] [deviceId=2] the Tsd recv data already [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.726.818 [devmm_svm.c:132][ascend][curpid:619427,619427][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.728.987 [devmm_virt_interface.c:450][ascend][curpid:619427,619427][drv][devmm][devmm_ioctl_init_process]Init_process details. (hostpid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.729.296 [drv_log_user.c:623][ascend][curpid:619427,619427][drv][devmm][share_log_read_in_single_module]Setup device succeeded. (logical_devid=2; devid=2; vfid=0; hostpid=619427; devpid=6117) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.729.335 [devmm_svm.c:122][ascend][curpid:619427,619427][drv][devmm][devmm_set_device_info]Device info. (devid=2; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.729.603 [stub_process_mode_nowin.cpp:60][ProcessQueueForMdc][tid:619427] [TsdClient] it is unnecessary of current mode[0] to grant queue auth to aicpusd [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.729.624 [stub_process_mode_nowin.cpp:99][OpenInHost][tid:619427] enter into OpenInHost deviceid[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.729.651 [client_manager.cpp:435][IsHostEnvironment][tid:619427] [TsdClient] logicDeviceId is [2], hostaicpunum[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.729.668 [stub_process_mode_nowin.cpp:103][OpenInHost][tid:619427] host cpu not support [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.729.682 [process_mode_manager.cpp:121][OpenProcess][tid:619427] [TsdClient][deviceId=2] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.729.701 [runtime.cc:2879] 619427 startAicpuExecutor: userDeviceId=2, devId=2, tsdOpen ret=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.729.717 [runtime.cc:2891] 619427 startAicpuExecutor: TsdOpen success, devId=2, ret=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.729.737 [runtime.cc:3627] 619427 DeviceRetain: new dev ok id=2, rawDevSize=1328 (bytes), ts_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.729.753 [raw_device.cc:936] 619427 DevSetTsId: DevSetTsId, tsId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.729.769 [npu_driver.cc:2968] 619427 DeviceOpen: Open device start, device_id=2, tsId=0. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.729.850 [trs_sqcq.c:458][ascend][curpid:619427,619427][drv][tsdrv][trsDevSqCqInit]Sqcq init. (devid=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.730.364 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=6; num=2044) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.730.864 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=7; num=2044) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.127 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=10; num=1024) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.379 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=11; num=1024) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.835 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=12; num=2048) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.860 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=8; num=16) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.882 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=9; num=16) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.903 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=13; num=4) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.731.923 [trs_sqcq.c:394][ascend][curpid:619427,619427][drv][tsdrv][trsIdTypeInit]Id type init success. (devId=2; tsId=0; id_type=14; num=10) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.734.171 [trs_dev_drv.c:302][ascend][curpid:619427,619427][drv][tsdrv][drvDeviceOpenComm]Open success. (devid=2; fd=36; mmapFd=37) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.734.197 [devmm_svm.c:132][ascend][curpid:619427,619427][drv][devmm][devmm_setup_device]DrvMemDeviceOpen. (devid=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.734.235 [devmm_svm.c:122][ascend][curpid:619427,619427][drv][devmm][devmm_set_device_info]Device info. (devid=2; dvpp_size=17179869184; support_bar_mem=1; support_dev_read_only=1; support_dev_mem_map_host=1; support_bar_huge_mem=1; is_alloced_by_malloc=0; host_rw_dev_ro=1; double_pgtable_offset=17592186044416; is_support_host_pin_pre_register=0; is_support_host_mem_pool=0) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.734.298 [npu_driver.cc:3011] 619427 DeviceOpen: Open device success, device_id=2, tsId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.734.319 [raw_device.cc:292] 619427 Init: chipType:0x5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.734.362 [raw_device.cc:304] 619427 Init: isAddrFlat:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.734.382 [raw_device.cc:320] 619427 Init: GetDevInfo, device_id=2, hardwareVersion=0x503 [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.734.399 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x1; out_feature=0x1) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.734.415 [npu_driver.cc:1578] 619427 SupportNumaTsMemCtrl: chip type=5, support flag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.735.357 [raw_device.cc:352] 619427 Init: Get phy info, phy chip id 2, phy die id 0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.735.379 [runtime.cc:2771] 619427 GetTsdQos: [TSD] start get qos, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.735.393 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.410 [tsd_client.cpp:189][TsdCapabilityGet][tid:619427] TsdCapabilityGet Begin. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.427 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.442 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.458 [process_mode_manager.cpp:1042][CapabilityGet][tid:619427] [ProcessModeManager] enter into CapabilityGet process deviceId[2] type[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.476 [process_mode_manager.cpp:386][InitTsdClient][tid:619427] [TsdClient] tsd client has been inited before [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.493 [version_verify.cpp:114][SpecialFeatureCheck][tid:619427] VersionVerify: previous type[38], supported [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.526 [process_mode_manager.cpp:1065][CapabilityGet][tid:619427] [TsdClient][deviceId=2] [sessionId=1] [type=0]send capability success [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.735.545 [hdc_client.cpp:154][TsdRecvData][tid:619427] recv timeout is:150000 ms [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.736.331 [process_mode_msg_parse.cpp:175][CapabilityResMsgProc][tid:619427] [CapabilityResMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.736.404 [process_mode_msg_parse.cpp:170][PidQosMsgProc][tid:619427] [TsdClient] PidQosMsgProc recvMsg realDeviceId[2] msgType[39] localDevId[2] rspCode[0],pidqos[1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.736.426 [hdc_client.cpp:172][TsdRecvData][tid:619427] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.736.447 [process_mode_manager.cpp:1070][CapabilityGet][tid:619427] [TsdClient][logicDeviceId_=2][type=0][pidQos=1][ret=0]recv capability respond finish [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.471 [runtime.cc:2782] 619427 GetTsdQos: [TSD] get qos success, userDeviceId=2, devId=2, qos=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.498 [engine.cc:78] 619427 Engine: Constructor. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.513 [stars_engine.cc:51] 619427 StarsEngine: Constructor. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.543 [engine_factory.cc:26] 619427 CreateEngine: new StarsEngine, Runtime_alloc_size 320 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.561 [engine_factory.cc:38] 619427 CreateEngine: Engine init success. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.576 [raw_device.cc:364] 619427 Init: new SpmPool ok, Runtime_alloc_size 56(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.591 [spm_pool.cc:42] 619427 Init: new buffer ok, Runtime_alloc_size 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.834 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.853 [spm_pool.cc:55] 619427 Init: new BufferAllocator ok, Runtime_alloc_size 152 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.867 [raw_device.cc:380] 619427 Init: new UmaArgLoader ok, Runtime_alloc_size 296(bytes) [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.736.885 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.903 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.736.917 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.736.932 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.744.464 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.744.503 [task_allocator.cc:46] 619427 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.744.544 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.744.560 [h2d_copy_mgr.cc:45] 619427 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.744.575 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.744.593 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.745.687 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 262144(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.745.711 [task_allocator.cc:46] 619427 DefaultAlloc: Runtime_alloc_size 4096(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.745.727 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 128(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.745.741 [h2d_copy_mgr.cc:45] 619427 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.745.754 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.745.771 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.575 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 524288(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.598 [task_allocator.cc:46] 619427 DefaultAlloc: Runtime_alloc_size 2048(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.613 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.639 [h2d_copy_mgr.cc:45] 619427 H2DCopyMgr: alloc h2d copy buff success, policy 2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.652 [uma_arg_loader.cc:86] 619427 Init: new BufferAllocator superArgAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.668 [task_allocator.cc:46] 619427 DefaultAlloc: Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.699 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.746.712 [uma_arg_loader.cc:93] 619427 Init: new BufferAllocator handleAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.747.571 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 131072(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.747.598 [uma_arg_loader.cc:102] 619427 Init: new BufferAllocator kernelInfoAllocator_ ok, Runtime_alloc_size 152 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.747.614 [uma_arg_loader.cc:104] 619427 Init: ALLOC PCIE is support[1] [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.747.630 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.747.649 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.190 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 32768(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.211 [h2d_copy_mgr.cc:45] 619427 H2DCopyMgr: alloc h2d copy buff success, policy 1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.227 [raw_device.cc:393] 619427 Init: new StreamSqCqManage ok, Runtime_alloc_size 416(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.243 [raw_device.cc:398] 619427 Init: New taskFactory ok, Runtime_alloc_size 136(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.261 [task_allocator.cc:46] 619427 DefaultAlloc: Runtime_alloc_size 393216(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.436 [buffer_allocator.cc:61] 619427 BufferAllocator: alloc success, Runtime_alloc_size 104(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.748.644 [raw_device.cc:109] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.018 [raw_device.cc:121] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.036 [raw_device.cc:425] 619427 Init: Runtime_alloc_size=64000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.090 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.170 [stream.cc:2246] 619427 AllocExecutedTimesSvm: stream_id=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.215 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.233 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.584 [npu_driver.cc:6125] 619427 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.633 [npu_driver.cc:6137] 619427 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1005, streamId=0, logicCq=1005, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.661 [npu_driver.cc:6189] 619427 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=0, logicCq=1005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.681 [npu_driver.cc:5916] 619427 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e9d0d000, addr=dfffe9d0d000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.709 [stream.cc:653] 619427 Setup: Success to get sq=0 sq reg virtual addr length=4096. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.764 [stream.cc:276] 619427 CreateStreamTaskRes: Normal STREAM Setup stream_id=0 flags_=0xc0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.782 [stream.cc:688] 619427 Setup: stream setup end, stream_id=0, IsTaskSink=0, sqId=0, cqId=0, deviceId=2, streamResId=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.801 [stream_state_callback_manager.cc:45] 619427 Notify: not reg stream state callback func! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.818 [raw_device.cc:593] 619427 Start: new primary Stream ok, Runtime_alloc_size 3872, stream_id=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.838 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.858 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=16 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.874 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.956 [tsch_stream.cc:45] 619427 Setup: stream setup end, stream_id=1, type=1, IsTaskSink=0, sqId=1, cqId=1, deviceId=2 streamResId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.749.976 [raw_device.cc:1219] 619427 TschStreamSetup: Tsch setup ffts dsa stream ok, stream_id=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.044 [osal.cc:309] 619427 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.168 [osal.cc:182] 619427 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=MONITOR_0, i=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.205 [osal.cc:309] 619427 CreateThread: Runtime_alloc_size 56 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.260 [stars_engine.cc:1785] 619586 MonitoringRun: Starts Monitor thread has entered, ts_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.292 [osal.cc:182] 619427 Start: mmCreateTaskWithThreadAttr, retCode=0, name_=RT_RECYCLE, i=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.305 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.310 [stars_engine.cc:201] 619587 RecycleThreadRun: RecycleThreadRun thread enter. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.750.320 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.750.347 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.750.361 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.381 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.425 [npu_driver.cc:6306] 619586 GetDeviceStatus: GetDeviceStatus status=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.449 [stars_engine.cc:1855] 619586 ReportHBMRasProc: g_hbmRasCheckDeviceId init, dev_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.465 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.644 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=0,task_type=97 (STARS_VERSION). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.670 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.705 [task_info.cc:1596] 619427 ConstructSqeForStarsVersionTask: Send GetStarsVersionTask succ,sqe_type=3,pre_p=1,sqe->rt_streamID=32768,stream_id=0,task_id=0,task_type=97. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.734 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.752 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=0, type=97(STARS_VERSION), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.784 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.800 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=0, task_type=97(STARS_VERSION). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.820 [stars_engine.cc:1122] 619427 SyncTask: Task Wait: device_id=2, stream_id=0, task_id=0, logicCqId=1005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.852 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.868 [stars_engine.cc:1071] 619427 IsSyncTaskFinish: Sync task finished, sqHead=0 finishedTaskId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.895 [stars_engine.cc:1017] 619427 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0x23, errType=0x20. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.915 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.930 [stars_engine.cc:1027] 619427 ProcReport: taskType=97. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.954 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.970 [stars_engine.cc:1570] 619427 ProcLogicCqReport: CQE warning, device_id=2, stream_id=0, task_id=0, sqe_type=3(place holder), errType=0x20(sq sw status error), sqSwStatus=0x23 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.750.991 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.009 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=0, task_type=97 (STARS_VERSION), task_finish_num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.027 [task_info.cc:1614] 619427 DoCompleteSuccessForStarsVersionTask: Complete StarsVersionTask succ, retCode=35,tschVersion=35. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.042 [raw_device.cc:1305] 619427 SetTschVersion: Set tsch version=35. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.074 [stars_engine.cc:1351] 619427 StarsResumeRtsq: Begin to query sq status, stream_id=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.093 [npu_driver.cc:5618] 619427 GetSqEnable: deviceId=2, tsId=0, sqId=0, enable=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.109 [stars_engine.cc:1365] 619427 StarsResumeRtsq: sq already disable, stream_id=0, queryCnt=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.122 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.137 [npu_driver.cc:5595] 619427 GetSqTail: deviceId=2, tsId=0, sqId=0, tail=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.153 [npu_driver.cc:5406] 619427 SetSqHead: deviceId=2, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.172 [npu_driver.cc:5357] 619427 EnableSq: deviceId=2, tsId=0, sqId=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.191 [stars_engine.cc:1430] 619427 StarsResumeRtsq: Resume stream_id=0, sq_id=0, sq_head=0, task_id=0, taskType=97, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.211 [npu_driver.cc:5618] 619427 GetSqEnable: deviceId=2, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.232 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.246 [raw_device.cc:1331] 619427 GetStarsVersion: send alloc get stars version task, stream_id=0, retCode:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.751.261 [raw_device.cc:610] 619427 Start: Get TschVersion, tschVersion=35. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.466 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=1,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.514 [ringbuffer_maintain_task.cc:98] 619427 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id=0, task_id=1, offset=0x53c0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.534 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=1, tail=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.550 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=1, type=83(DEVICE_RINGBUFFER_CONTROL), head=1, tail=2, sqe_num=1, lastTail=1, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.568 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.583 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=1, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.599 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.617 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.647 [stars_engine.cc:622] 619427 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=1005, retCode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.665 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.681 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.695 [stream.cc:2886] 619427 StarsWaitForTask: No.1 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.707 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.721 [stream.cc:2886] 619427 StarsWaitForTask: No.2 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.735 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.747 [stream.cc:2886] 619427 StarsWaitForTask: No.3 stream_id=0, taskId=1, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.760 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.773 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.788 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.801 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=1, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.820 [stars_engine.cc:437] 619427 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=1, sqHead=2, endTaskId=1, recycleTaskId=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.836 [stream.cc:2886] 619427 StarsWaitForTask: No.4 stream_id=0, taskId=1, currentId=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.889 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=2,task_type=29 (ALLOC_DSA_ADDR). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.917 [task_info.cc:111] 619427 ConstructSqeForAllocDsaAddrTask: Send AllocDsaAddrTask succ, sqe_type=3, pre_p=1, stream_id=0, task_id=2, task_type=29, dsa_sq_id=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.935 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), head=2, tail=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.950 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=2, type=29(ALLOC_DSA_ADDR), head=2, tail=3, sqe_num=1, lastTail=2, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.969 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.752.987 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=2, task_type=29(ALLOC_DSA_ADDR). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.002 [stars_engine.cc:1122] 619427 SyncTask: Task Wait: device_id=2, stream_id=0, task_id=2, logicCqId=1005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.116 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.132 [stars_engine.cc:1071] 619427 IsSyncTaskFinish: Sync task finished, sqHead=3 finishedTaskId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.144 [stars_engine.cc:1017] 619427 ProcReport: Get logic report: cnt=1, idx=0, stream_id=0, report task_id=2, sync task_id=2, sqeType=3, sqeHead=2, retCode=0, errType=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.162 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.173 [stars_engine.cc:1027] 619427 ProcReport: taskType=29. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.184 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.197 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.213 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=2, task_type=29 (ALLOC_DSA_ADDR), task_finish_num=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.232 [npu_driver.cc:5618] 619427 GetSqEnable: deviceId=2, tsId=0, sqId=0, enable=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.248 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.753.259 [raw_device.cc:1232] 619427 TschStreamAllocDsaAddr: send alloc dsa sq task, stream_id=1, retCode:0 [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.753.280 [atrace_api.c:40](tid:619427) AtraceCreate start. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.395 [trace_rb_log.c:45](tid:619427) init ring buffer time finished, realTime 1743644143753334632, monotonicTime 4211296327691, freq 99994 kHz, minutesWest 480 minutes [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.414 [trace_rb_log.c:137](tid:619427) [RUNTIME_ATRACE_DEV2_TS0] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.432 [tracer_mgr_operate.c:66](tid:619427) create object RUNTIME_ATRACE_DEV2_TS0 success, exitSave(false), noLock(false). [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.445 [atrace_api.c:44](tid:619427) AtraceCreate end. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.496 [trace_event.c:70](tid:619427) limitedNum 1, processType 0 [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.754.513 [trace_event.c:372](tid:619427) set trace event attr limitedNum: 1, processType: 0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.532 [runtime.cc:3159] 619427 SetTsdProfCallback: set tsd prof callback [INFO] TDT(619427,python3.9):2025-04-03-09:35:43.754.559 [client_manager.cpp:196][SetProfilingCallback][tid:619427] [TsdClient] set profiling callback success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.590 [runtime.cc:3364] 619427 PrimaryContextRetain: new Context ok, Runtime_alloc_size 424 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.615 [context.cc:109] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 160000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.669 [context.cc:121] 619427 Init: ObjAllocator alloc success, Runtime_alloc_size 960000(bytes) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.685 [context.cc:276] 619427 Init: Runtime_alloc_size is 16000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.762 [raw_device.cc:803] 619427 UpdateTimeoutConfig: No need to update timeout config. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.780 [npu_driver_dcache_lock.cpp:39] 619427 AllocAddrForDcache: halMemCtl failed, drvRet=65534, outSize=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.885 [raw_device.cc:528] 619427 AllocStackPhyBase: AllocStackPhyBase device_id=2, stackPhyBase32k_=0x12c140000000, stackPhySize=2457600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.934 [raw_device.cc:232] 619427 LaunchDcacheLockOp: dcache lock has been set or stack addr is not for dcache, dCacheLockFlag_=0, stackAddrIsDcache_=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.959 [runtime.cc:3393] 619427 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.754.976 [runtime.cc:3560] 619427 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.755.050 [api_impl.cc:2987] 619427 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.755.068 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.755.083 [api_impl.cc:2992] 619427 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:43.755.107 [awatchdog_core.c:175](tid:619427) create watchdog for id : 65543, tid : 619427, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.755.124 [api_impl.cc:3025] 619427 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.755.151 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.755.168 [acl.cpp:121]619427 GetSocVersion: socVersion is Ascend910B3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.436 [api_impl.cc:3476] 619427 GetDeviceInfo: get device info, deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.469 [api_impl.cc:3476] 619427 GetDeviceInfo: get device info, deviceId=2 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.758.522 [device.cpp:133]619427 UpdatePlatformInfoWithDevice: Successfully to UpdatePlatformInfoWithDevice, deviceId = 2, aicCnt = 20, vecCoreCnt = 40, cubeCoreCnt = 20 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.758.545 [device.cpp:154]619427 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.758.565 [context.cpp:87]619427 aclrtGetCurrentContext: start to execute aclrtGetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.758.583 [context.cpp:98]619427 aclrtGetCurrentContext: successfully execute aclrtGetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.758.608 [stream.cpp:48]619427 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.639 [api_impl.cc:1120] 619427 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.716 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.750 [stream.cc:2246] 619427 AllocExecutedTimesSvm: stream_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.789 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.807 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.758.994 [npu_driver.cc:6125] 619427 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.046 [npu_driver.cc:6137] 619427 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1006, streamId=2, logicCq=1006, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.069 [npu_driver.cc:6189] 619427 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=2, logicCq=1006. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.087 [npu_driver.cc:5916] 619427 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e01c8000, addr=dfffe01c8000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.102 [stream.cc:653] 619427 Setup: Success to get sq=2 sq reg virtual addr length=4096. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.759.153 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.173 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.759.330 [stream.cc:291] 619427 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=2, flags_=0x600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.753 [stream.cc:688] 619427 Setup: stream setup end, stream_id=2, IsTaskSink=0, sqId=2, cqId=2, deviceId=2, streamResId=65535 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.781 [stream_state_callback_manager.cc:45] 619427 Notify: not reg stream state callback func! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.797 [api_impl.cc:1162] 619427 StreamCreate: Succ, flags=1536, stream_id=2, context=aaaae7011ab0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.760.817 [stream.cpp:189]619427 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.847 [stream.cc:3161] 619427 SetFailMode: stream_id=2 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.760.863 [stream.cpp:195]619427 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.760.903 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaae80610a0. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.760.931 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:377] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.760.947 [stream.cpp:48]619427 aclrtCreateStreamWithConfig: start to execute aclrtCreateStreamWithConfig with priority:0, flag:3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.966 [api_impl.cc:1120] 619427 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.760.997 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.030 [stream.cc:2246] 619427 AllocExecutedTimesSvm: stream_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.068 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.095 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.256 [npu_driver.cc:6125] 619427 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.308 [npu_driver.cc:6137] 619427 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1007, streamId=3, logicCq=1007, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.331 [npu_driver.cc:6189] 619427 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=3, logicCq=1007. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.347 [npu_driver.cc:5916] 619427 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e01a7000, addr=dfffe01a7000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.358 [stream.cc:653] 619427 Setup: Success to get sq=3 sq reg virtual addr length=4096. [INFO] DRV(619427,python3.9):2025-04-03-09:35:43.761.403 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.423 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.761.726 [stream.cc:291] 619427 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=3, flags_=0x600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.370 [stream.cc:688] 619427 Setup: stream setup end, stream_id=3, IsTaskSink=0, sqId=3, cqId=3, deviceId=2, streamResId=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.391 [stream_state_callback_manager.cc:45] 619427 Notify: not reg stream state callback func! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.406 [api_impl.cc:1162] 619427 StreamCreate: Succ, flags=1536, stream_id=3, context=aaaae7011ab0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.422 [stream.cpp:189]619427 aclrtSetStreamFailureMode: start to execute aclrtSetStreamFailureMode, mode is 1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.436 [stream.cc:3161] 619427 SetFailMode: stream_id=3 is updated from CONTINUE to STOP. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.449 [stream.cpp:195]619427 aclrtSetStreamFailureMode: successfully execute aclrtSetStreamFailureMode, mode is 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.762.482 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:105] RegCallback] Register callback thread, stream : 0xaaaae8191fc0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.605 [callback.cpp:25]619427 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471067484064. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.642 [npu_driver.cc:462] 619427 EventIdAlloc: event id alloc begin, device_id=2, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.647 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.682 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.702 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.707 [npu_driver.cc:483] 619427 EventIdAlloc: Event id alloc success, device_id=2, ts_id=0, event_id=0, eventFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.726 [event.cc:347] 619427 GenEventId: Event id alloc success, device_id=2, tsId=0, eventFlag=1, event_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.746 [subscribe.cc:54] 619427 AssignGroupId: give groupId=576 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.763 [npu_driver.cc:3775] 619427 SqCqAllocate: deviceId=2, tsId=0, groupId=576. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.781 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.796 [npu_driver.cc:3785] 619427 SqCqAllocate: alloc success, deviceId=2 sqId=458, cqId=458. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.813 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.814 [subscribe.cc:131] 619427 Insert: callback subscribe: groupId=576, threadId=281471067484064, devId=2, tsId=0, streamId=2, sqId=458, cqId=458 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.839 [callback.cpp:31]619427 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471067484064. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.847 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.762.868 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaae80610a0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.956 [callback.cpp:25]619427 aclrtSubscribeReport: start to execute aclrtSubscribeReport, threadId is 281471059029920. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.762.978 [npu_driver.cc:462] 619427 EventIdAlloc: event id alloc begin, device_id=2, ts_id=0, remote_event_id=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.762.991 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.000 [npu_driver.cc:483] 619427 EventIdAlloc: Event id alloc success, device_id=2, ts_id=0, event_id=1, eventFlag=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.025 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.026 [event.cc:347] 619427 GenEventId: Event id alloc success, device_id=2, tsId=0, eventFlag=1, event_id=1 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.040 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.050 [subscribe.cc:54] 619427 AssignGroupId: give groupId=577 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.763.054 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.062 [npu_driver.cc:3775] 619427 SqCqAllocate: deviceId=2, tsId=0, groupId=577. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.763.071 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.080 [npu_driver.cc:3785] 619427 SqCqAllocate: alloc success, deviceId=2 sqId=459, cqId=459. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.095 [subscribe.cc:131] 619427 Insert: callback subscribe: groupId=577, threadId=281471059029920, devId=2, tsId=0, streamId=3, sqId=459, cqId=459 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.763.115 [callback.cpp:31]619427 aclrtSubscribeReport: successfully execute aclrtSubscribeReport, threadId is 281471059029920. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.120 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.143 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:125] RegCallback] Register callback thread success, stream : 0xaaaae8191fc0. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.167 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:386] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.250 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:110] Initialize] Config huge_page_reserve_size : 0, device_hbm_huge_page_reserved_size_ : 0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.585 [npu_driver.cc:2616] 619427 MemGetInfoEx: memory info type=1, type=2, free=8857477120, hug_free=56245616640 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.629 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:60] AscendVmmAdapter] VMM align size is 2097152 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.694 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:158] CheckVmmDriverVersion] Driver path is /usr/local/Ascend [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.738 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:187] CheckVmmDriverVersion] Driver version is 25.0.rc1.b070, major version is 25 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.768 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.h:113] IsVmmEnabled] VMM is enabled. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.789 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:180] Initialize] Device MOC Size:62420M, Device free MOC Size:62087M, Reserved MOC size for Other Components(HCCL/rts/etc.):3891M, Recommend Reserved MOC size for Other Components:3880M, User define MindSpore MOC Size:0G, MindSpore Used MOC Size:58196M. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.763.812 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_dynamic_mem_adapter.cc:57] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58196M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 0M [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.763.841 [event.cpp:260]619427 aclrtSetOpWaitTimeout: start to execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.763.863 [api_impl.cc:5713] 619427 SetOpWaitTimeOut: set OP wait timeout:1980. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.764.939 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=3,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.764.962 [timeout_set_task.cc:115] 619427 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.764.980 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), head=3, tail=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.764.997 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=3, type=86(TASK_TIMEOUT_SET), head=3, tail=4, sqe_num=1, lastTail=3, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.017 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.033 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=3, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.048 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.065 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.096 [stars_engine.cc:622] 619427 ProcLogicCqUntilEmpty: Task Wait: stream_id=0, logicCqId=1005, retCode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.113 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=0, taskId=3, currentId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.139 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.153 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.166 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.181 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=3, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.200 [stars_engine.cc:437] 619427 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=3, sqHead=4, endTaskId=3, recycleTaskId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.218 [stream.cc:2886] 619427 StarsWaitForTask: No.1 stream_id=0, taskId=3, currentId=3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.765.231 [event.cpp:262]619427 aclrtSetOpWaitTimeout: successfully execute aclrtSetOpWaitTimeout, timeout = 1980s [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.765.248 [event.cpp:269]619427 aclrtSetOpExecuteTimeOut: start to execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.266 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=4,task_type=86 (TASK_TIMEOUT_SET). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.282 [timeout_set_task.cc:115] 619427 ConstructSqeForTimeoutSetTask: TaskTimeoutSetTask ConstructSqe finish,topic_type=0,cmd_type=13 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.296 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), head=4, tail=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.311 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=4, type=86(TASK_TIMEOUT_SET), head=4, tail=5, sqe_num=1, lastTail=4, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.331 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.348 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=4, task_type=86(TASK_TIMEOUT_SET). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.364 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.379 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.392 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.408 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.420 [stream.cc:2886] 619427 StarsWaitForTask: No.1 stream_id=0, taskId=4, currentId=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.434 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.449 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.460 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.472 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=4, task_type=86 (TASK_TIMEOUT_SET), task_finish_num=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.489 [stars_engine.cc:437] 619427 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=4, sqHead=5, endTaskId=4, recycleTaskId=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.506 [stream.cc:2886] 619427 StarsWaitForTask: No.2 stream_id=0, taskId=4, currentId=4 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.765.519 [event.cpp:277]619427 aclrtSetOpExecuteTimeOut: successfully execute aclrtSetOpExecuteTimeOut, timeout = 900s [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.552 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:285] Init] End inititializing kernel runtime. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.587 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:48] InitDevice] Enter SetRtDevice, current initialize device number:1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.606 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:50] InitDevice] Device 2 has been set [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.624 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:381] CreateDefaultStream] The default compute stream is already created, skip. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.642 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:390] CreateDefaultStream] The default communication stream is already created, skip. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.690 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_pool.cc:165] DefaultEnhancedAscendMemoryPool] DefaultEnhancedAscendMemoryPool constructed. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.728 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:397] Initialize] Skip initialization of memory pool since init size is not configured. [INFO] UTILS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:43.765.783 [mindspore/ccsrc/utils/offload_context.cc:86] enable_aio] On ascend devices, enable aio and enable pinned mem cannot be turned on at the same time. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.765.905 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.765.928 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.765.972 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.766.004 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:43.766.021 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.766.034 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.766.193 [acl.cpp:288]619427 aclInit: start to execute aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.766.229 [acl.cpp:298]619427 aclInit: call ErrorManager.Initialize [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.766.334 [acl.cpp:242]619427 HandleErrorManagerConfig: err_msg mode is set ["1"]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.775.318 [error_manager.cc:673]619427 ParseJsonFile:Parse json file:/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/../conf/error_manager/error_code.json success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:43.775.904 [acl.cpp:315]619427 aclInit: call ge interface executor.Initialize [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:43.775.974 [prof_atls_plugin.cpp:210] (tid:619427) Module[45] register callback of ctrl handle. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.776.016 [ge_executor.cc:273]619427 Initialize:Init GeExecutor begin. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.776.276 [op_lib_register.cc:117]619427 GetAllCustomOpApiSoPaths:value of env ASCEND_CUSTOM_OPP_PATH is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ho [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.382 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.422 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.458 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.491 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.521 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.551 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.582 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.612 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.641 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.671 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.701 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.730 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.767 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.799 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.830 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.859 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.889 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.918 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.950 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/libcust_opapi.so, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.776.980 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/libcust_opapi.so, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.019 [op_lib_register.cc:165]619427 CallInitFunc:CallInitFunc 0 successfully, op_lib_paths_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/je [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.057 [plugin_manager.cc:616]619427 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.074 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.104 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.129 [plugin_manager.cc:624]619427 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.146 [plugin_manager.cc:206]619427 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619427,python3.9):2025-04-03-09:35:43.777.912 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.777.955 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.777.973 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.777.986 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.032 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.382 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.417 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.433 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.446 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.482 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.524 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.542 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.559 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.574 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.602 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.778.944 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.979 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.778.996 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.009 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.046 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.389 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.422 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.439 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.451 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.487 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.522 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.539 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.555 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.568 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.585 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.779.925 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.958 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.984 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.779.997 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.032 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.381 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.420 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.438 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.451 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.485 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.520 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.537 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.551 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.569 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.583 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.780.919 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.951 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.970 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.780.985 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.022 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.362 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.407 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.424 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.438 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.475 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.511 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.527 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.540 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.556 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.573 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.781.911 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.943 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.960 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.781.972 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.006 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.347 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.379 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.395 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.409 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.455 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.492 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.508 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.782.521 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.537 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.551 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.567 [plugin_manager.cc:218]619427 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.586 [plugin_manager.cc:422]619427 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.600 [plugin_manager.cc:155]619427 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.623 [plugin_manager.cc:158]619427 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.639 [plugin_manager.cc:426]619427 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.782.969 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.783.001 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.783.018 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.783.034 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.084 [plugin_manager.cc:439]619427 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.106 [plugin_manager.cc:175]619427 ReversePathString:Enter ReversePathString schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.128 [plugin_manager.cc:187]619427 ReversePathString:path_str is '/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/ [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.450 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.481 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:43.783.503 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.783.551 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/] failed [INFO] GE(619427,python3.9):2025-04-03-09:35:43.799.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.799.984 [op_tiling_registry.cc:162]619427 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:Aipp [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.105 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:AvgPool1DD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.225 [op_tiling_registry.cc:162]619427 OpTilingFuncRegistry:Register op tiling and parse function V4 for op_type:AutoTiling [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.333 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Bias [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.434 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:INTrainingUpdateV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.530 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:CheckValid [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.657 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.679 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ConcatV2D [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.698 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Pack [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.794 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:ConcatOffset [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.888 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Diag [INFO] GE(619427,python3.9):2025-04-03-09:35:43.800.980 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:DiagPart [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.077 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DynSeqOuter [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.175 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Expand [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.500 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Ger [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.591 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:IFMR [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.702 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:InTopKD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.804 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:KlDivLossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.801.902 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:L1LossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.005 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNorm [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.112 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormBetaGammaBackpropV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.297 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:LayerNormXBackprop [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.399 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.508 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LayerNormXBackpropV3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.609 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduce [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.628 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:LpNormReduceV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.731 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:LRUCacheV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.840 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.859 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:MaxPoolGradWithArgmaxV1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.802.958 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:MaxPoolWithArgmaxV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.068 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MovingSumWithSigmoid [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.171 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:MseLossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.373 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:NMSWithMask [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.545 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.644 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:PadV3Grad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.747 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReduceStdWithMean [INFO] GE(619427,python3.9):2025-04-03-09:35:43.803.854 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Resize [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.285 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ResizeBilinearV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.397 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:ReverseV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.513 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Scale [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.694 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLoss [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.807 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SigmoidFocalLossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.804.927 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsPack [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.035 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SignBitsUnpack [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.136 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossGradV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.234 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SmoothL1LossV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.337 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftMarginLossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.453 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SoftmaxFocalLossGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.553 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:Sort [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.573 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:SortV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.677 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseApplyFtrlD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.784 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SparseReshape [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.885 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SplitV [INFO] GE(619427,python3.9):2025-04-03-09:35:43.805.918 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Split [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.022 [op_tiling_registry.cc:122]619427 OpTilingFuncRegistry:Register op tiling function V1 for op_type:SquareSumAll [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.127 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SyncBatchNormGatherStats [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.229 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Tile [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.333 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileD [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.447 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:TileWithAxis [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.642 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:Unpack [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.738 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.834 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.914 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.806.994 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.076 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.151 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.178 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.270 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.300 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.326 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.351 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.478 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.555 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.632 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.707 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.873 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.912 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.807.991 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.019 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.046 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.225 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.307 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.385 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.460 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.536 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.699 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.777 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.854 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.808.957 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.039 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.123 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.281 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.809.437 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.813.587 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:DepthToSpace [INFO] GE(619427,python3.9):2025-04-03-09:35:43.813.618 [op_tiling_registry.cc:148]619427 OpTilingFuncRegistry:Register op tiling and parse function V3 for op_type:SpaceToDepth [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.035 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.165 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.559 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.693 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.805 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.815.927 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.085 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.209 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.368 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.518 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.636 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.749 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.816.876 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.001 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.116 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.149 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.307 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.418 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.559 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.680 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.799 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.817.963 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.078 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.235 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.354 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.830 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.818.950 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.068 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.187 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.305 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.337 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.550 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.674 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.819.824 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.016 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.134 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.518 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.629 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.659 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.791 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.819 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.820.934 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.073 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.189 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.299 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.412 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.529 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.737 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.767 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.877 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.821.990 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.303 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.424 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.535 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.654 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.822.854 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.165 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.328 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.461 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.586 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.704 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.815 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.823.929 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.409 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.465 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffefd345d70, registered count:1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.662 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffefd34e248, registered count:2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.791 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.824.907 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.053 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.170 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.296 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.416 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.649 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.774 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.825.916 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.144 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.266 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.389 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.506 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.630 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.739 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.826.860 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.011 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.040 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.067 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.093 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.362 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.478 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.587 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.699 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.809 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.837 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.864 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.827.987 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.099 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.544 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.655 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.770 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.828.902 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.016 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.137 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.166 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.394 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.707 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.819 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.829.929 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.049 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.159 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.397 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.511 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.710 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.831 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.830.954 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.833.551 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.833.600 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.833.632 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.833.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.833.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.021 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.137 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.250 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.487 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.604 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.729 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.886 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.834.915 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.039 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.068 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.174 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.835.752 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.836.030 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.836.199 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.836.322 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.836.448 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.836.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.837.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.838.043 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.838.358 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.838.810 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.839.401 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.839.524 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.839.646 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.839.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.839.920 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.840.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.842.436 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.843.267 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.843.436 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.843.593 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.844.585 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.844.741 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.844.902 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.589 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.818 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.845.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.065 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.175 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.401 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.687 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.804 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.846.923 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.111 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.343 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.456 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.604 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.777 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.847.896 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.013 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.040 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.237 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.479 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.657 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.772 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.848.934 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.046 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.294 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.322 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.435 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.463 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.576 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.689 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.840 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.849.956 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.075 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.262 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.387 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.590 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.742 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.850.924 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.299 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.597 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.726 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.840 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.960 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.851.996 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.311 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.433 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.550 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.666 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.776 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.888 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.852.919 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.853.039 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.853.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.854.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.856.995 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.857.203 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.857.410 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.857.722 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.857.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.858.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.858.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.055 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.336 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.503 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.650 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.807 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.859.969 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.271 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.395 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.796 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.830 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.884 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.908 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.933 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.958 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.860.984 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.060 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.088 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.140 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.165 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.204 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.255 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.329 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.351 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.373 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.399 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.424 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.448 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.497 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.545 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.570 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.598 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.623 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.648 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.676 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.727 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.751 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.787 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.813 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.838 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.863 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.888 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.912 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.962 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.861.988 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.012 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.041 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.067 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.094 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.120 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.144 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.169 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.194 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.218 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.553 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.728 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.756 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.781 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.814 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.840 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.862.993 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.109 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.225 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.464 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.578 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.863.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.864.036 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.340 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.489 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.548 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.674 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.703 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.730 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.865.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.015 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.193 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.345 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.469 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.671 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.866.794 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.001 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.128 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.246 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.303 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.649 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.811 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.867.964 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.171 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.329 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.629 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.798 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.868.984 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.127 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.246 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.366 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.494 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.614 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.869.957 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.870.076 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.870.194 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.870.316 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.870.450 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.870.565 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.028 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.493 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.619 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.740 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.769 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.871.896 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.016 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.137 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.285 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.690 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.819 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.872.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.356 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.697 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.831 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.873.951 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.156 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.471 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.588 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.715 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.874.968 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.096 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.378 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.500 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.668 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.875.937 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.876.147 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.876.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.876.529 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.876.759 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.876.987 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.260 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.285 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.307 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.341 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.366 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.391 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.877.416 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.878.200 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.878.629 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.879.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.879.256 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.879.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.879.898 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.880.035 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.880.232 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.882.998 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.884.843 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.884.992 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.885.117 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.885.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.885.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.885.698 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.885.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.248 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.308 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.334 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.361 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.588 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.886.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.107 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.277 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.480 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.680 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.887.855 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.029 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.399 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.567 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.595 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.957 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.888.987 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.104 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolGrad [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.139 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.261 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.290 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.420 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.448 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.573 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.699 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.830 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.889.967 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.890.080 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.890.193 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.890.398 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.890.618 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.891.001 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.891.395 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.891.606 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.891.939 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.348 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.470 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPool [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.506 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.618 [op_tiling_registry.cc:134]619427 OpTilingFuncRegistry:Register op tiling function V2 for op_type:AvgPoolV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.651 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.808 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.839 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.870 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.892.899 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.894.059 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.894.535 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.894.694 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.894.851 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.041 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.338 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.486 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.684 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.895.845 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.896.000 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.896.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.896.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.896.768 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.897.130 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.897.369 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.897.568 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.898.648 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.899.096 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.899.665 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.899.829 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.899.988 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.114 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.301 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.609 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.640 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.900.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.901.106 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.901.269 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.901.432 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.901.563 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.901.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.075 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.359 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.474 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.644 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.902.831 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.211 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.239 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.415 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.903.605 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.905.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.910.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.910.997 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.913.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.914.159 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.914.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.914.793 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.914.997 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.367 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.526 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.713 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.861 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.915.959 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.916.234 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.916.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.916.528 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.010 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.178 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.338 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.374 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.490 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.917.809 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.008 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [Tiling][InputsDependency][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.208 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.385 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.509 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.625 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.918.656 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.919.359 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.919.579 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.919.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.919.854 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.205 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.329 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.387 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.534 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.742 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.920.835 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.316 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.346 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.544 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.921.577 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.412 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.503 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.530 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DTranspose, registered count: 3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.602 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2D, registered count: 4 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.680 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 5 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.709 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3DTranspose, registered count: 6 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.782 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3D, registered count: 7 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.940 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: FFN, registered count: 8 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.924.960 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 9 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.075 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulV2, registered count: 10 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.098 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 11 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.128 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulV3, registered count: 12 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.150 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 13 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.328 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: DynamicRNN, registered count: 14 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.925.347 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.926.540 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffefdc98a10, registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.926.573 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffefdc98c70, registered count:1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.926.990 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.927.474 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.929.213 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [Tiling][GenSimplifiedKey][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.929.376 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.929.460 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.929.863 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.930.320 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.961.934 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.962.071 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.962.277 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.962.368 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.962.514 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.962.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.963.312 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.963.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.963.683 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.963.775 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.963.935 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [Tiling][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.964.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [HostInputs][OpExecuteFunc] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.964.368 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [Tiling][TilingDependency][TilingDependencyPlacement][TilingParse] [INFO] GE(619427,python3.9):2025-04-03-09:35:43.964.892 [op_tiling_manager.cc:79]619427 LoadSo:dlopen file /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.964.968 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.372 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.425 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.479 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.547 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.589 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.632 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.965.680 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.966.693 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.966.729 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.966.754 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.966.777 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.966.802 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:1 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.156 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff2c0999b0, registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.184 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:5 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.205 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.224 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.243 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:2 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.566 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff2c09cda0, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.592 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.613 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.633 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.652 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.967.967 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.968.021 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.968.111 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.968.901 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff2c0532c0, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.968.933 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.968.956 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.968.975 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.968.992 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.318 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff2c057978, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.343 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.363 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.381 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.399 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.969.696 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.969.746 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.969.833 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.969.891 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.969.943 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.969.993 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.031 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.070 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.112 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.145 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.183 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.222 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.261 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.298 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.338 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.379 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.418 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.458 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.491 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.529 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.567 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.606 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.643 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.681 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.713 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.751 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.799 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.970.833 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.871 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.909 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.947 [op_tiling_manager.cc:73]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/liboptiling.so: cannot open shared object file: No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.970.984 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.971.026 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.060 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.971.098 [op_tiling_manager.cc:58]619427 LoadSo:[FindSo][Check] Get path with op_master [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/] failed [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.971.137 [op_tiling_manager.cc:67]619427 LoadSo:Failed to dlopen /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so! errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/liboptiling.so: cannot open shared object file: No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.177 [op_tiling_manager.cc:75]619427 LoadSo:dlopen file /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/liboptiling.so successfully! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.198 [op_tiling_manager.cc:109]619427 ~FuncPerfScope:[GEPERFTRACE] The time cost of OpTilingManager::LoadSo is [194142] micro seconds. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.314 [plugin_manager.cc:639]619427 GetConstantFoldingOpsPath:Enter GetConstantFoldingOpsPath schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.333 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.358 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:43.971.417 [host_cpu_engine.cc:254]619427 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so [INFO] GE(619427,python3.9):2025-04-03-09:35:43.977.048 [host_cpu_engine.cc:273]619427 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libconstant_folding_ops.so has been opened [INFO] GE(619427,python3.9):2025-04-03-09:35:43.977.142 [host_cpu_engine.cc:254]619427 LoadLib:To invoke dlopen on lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] GE(619427,python3.9):2025-04-03-09:35:43.983.624 [host_cpu_engine.cc:267]619427 LoadLib:Invoke function Initialize in lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so [INFO] OP(619427,python3.9):2025-04-03-09:35:43.984.542 [host_cpu_kernel_wrapper.cpp:88][OP_PROTO][Initialize][619427] OpName:[HostCpuKernelWrapperOp] "Host cpu dlopen so[/usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libaicpu_const_folding.so] success" [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.582 [folding.cc:404][InitCpuConstantFoldingNew][tid:619427]:"Init cpu constant folding begin." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.669 [folding.cc:408][InitCpuConstantFoldingNew][tid:619427]:"Number of registered ops: 234" [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.691 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Acos]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.717 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Acosh]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.735 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ActiveRotatedFilter]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.753 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ActiveRotatedFilterGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.771 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[AdaptiveAvgPool2dAssistMatrix]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.789 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[AdaptiveAvgPoolAssistMatrix]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.805 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[AdaptiveMaxPool2d]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.820 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Add]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.837 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Addcdiv]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.854 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Addcmul]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.870 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[AffineGrid]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.886 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ApplyAdamV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.918 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Asin]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.937 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Asinh]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.957 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Assert]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.972 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Atan]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.984.989 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Atan2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.007 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Atanh]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.023 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[AvgPoolV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.038 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Betainc]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.052 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Bucketize]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.067 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CTCLossV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.084 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CTCLossV2Grad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.102 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CacheSwapTable]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.119 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CalcBucketsLimitAndOffset]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.136 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CaseCondition]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.150 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cast]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.167 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Ceil]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.181 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CheckNumerics]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.198 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ClipByValue]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.215 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ClipByValueV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.230 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Combinations]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.243 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CompareAndBitpack]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.259 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ComputeAccidentalHits]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.274 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ConcatV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.290 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Conj]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.307 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ConjugateTranspose]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.325 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Coordinates1DTo2D]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.339 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cosh]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.356 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[CropAndResize]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.373 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cross]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.394 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cummax]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.410 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cummin]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.427 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Cumsum]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.444 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Div]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.458 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[DynamicStitch]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.473 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EditDistance]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.486 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Elu]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.499 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EluGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.515 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplyAdaGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.529 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplyAdam]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.544 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplyAdamW]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.557 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplyFtrl]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.572 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplyRmsprop]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.586 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingApplySgd]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.602 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingComputeVarExport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.621 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingComputeVarImport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.635 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingFeatureMappingExport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.650 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingFeatureMappingFileSize]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.666 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingFeatureMappingImport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.680 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingHashmapExport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.696 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingHashmapFileSize]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.710 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingHashmapImport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.725 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingLookup]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.739 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingTableEvict]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.753 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingTableExport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.769 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingTableFind]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.782 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingTableFindAndInit]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.798 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[EmbeddingTableImport]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.820 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Equal]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.835 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Expand]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.848 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ExpandDims]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.864 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Exponential]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.884 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ExponentialDecayLR]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.901 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Fill]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.919 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[FillDiagonal]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.935 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Fills]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.952 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Floor]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.964 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[FloorDiv]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.977 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[FloorMod]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.985.992 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GEMM]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.009 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GatherElements]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.026 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GatherV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.039 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Geometric]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.055 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GetDynamicDims]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.067 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Greater]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.082 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GreaterEqual]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.099 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GridSampler2D]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GridSampler2DGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.129 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GridSampler3D]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.141 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[GridSampler3DGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.157 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Histogram]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.169 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[IMGWarpOffsets]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.185 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Identity]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.198 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[IndexToAddr]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.214 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InitEmbeddingHashmap]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.231 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InitPartitionMap]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.252 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InplaceIndexAdd]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.265 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InplaceTopKDistance]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.281 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InvGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.298 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[InvertPermutation]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.313 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[IsClose]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.331 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[IsInf]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.348 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[IsNan]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.364 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Less]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.376 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[LinSpace]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.391 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[LinearSumAssignment]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.404 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Log]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.420 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Log1p]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.433 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[LogMatrixDeterminant]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.448 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[LogSoftmaxV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.463 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[LogUniformCandidateSampler]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.478 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MaskedFill]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.491 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MaskedScatter]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.506 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MaskedSelect]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.519 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MaskedSelectGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.533 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MatMul]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.547 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MatrixDiagPartV3]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.562 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MatrixDiagV3]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.580 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MinAreaPolygons]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.593 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Mul]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.609 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MultinomialAliasDraw]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.622 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MultinomialAliasSetup]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.640 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[MultinomialWithReplacement]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.656 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NLLLoss]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.671 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NLLLossGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.693 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NMSWithMask]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.710 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Neg]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.726 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NonMaxSuppressionV3]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.741 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NonZero]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.754 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NonZeroWithValueShape]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.769 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NonZeroWithValueShapeV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.785 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[NotEqual]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.801 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[OnesLike]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.818 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[PSAMask]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.833 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[PSAMaskGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.850 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Pad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.866 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[PadD]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.881 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[PadV3]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.897 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[PadV3Grad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.912 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Pinverse]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.928 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Poisson]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.947 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Pow]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.964 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ProdEnvMatACalcRij]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.979 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RNNTLoss]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.986.995 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RandomStandardNormal]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.010 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RandomUniform]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.024 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RealDiv]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.040 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ReduceAll]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.053 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ReduceMean]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.069 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ReduceSum]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.085 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Reshape]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.098 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Resize]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.114 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ResizeBilinear]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.138 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ResizeBilinearGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.153 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ResizeBilinearV2Grad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.168 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ResizeGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.185 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ResizeTrilinear]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.202 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ReverseSequence]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.219 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RightShift]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.236 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RoiPoolingWithArgMax]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.256 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[RotatedNMS]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.270 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Round]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.286 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Rsqrt]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.302 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[STFT]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.319 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ScatterAddWithAxis]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.333 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ScatterElements]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.349 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ScatterNdAdd]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.363 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ScatterNdMax]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.379 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ScatterNdMin]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.393 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SearchSorted]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.406 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SelectV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.419 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Sigmoid]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.433 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SigmoidGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.448 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Sign]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.464 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SilentCheck]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.481 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SilentCheckV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.498 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SilentCheckV3]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.512 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SliceWrite]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.525 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SoftmaxV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.541 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Sort]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.558 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SparseFillEmptyRows]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.575 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SparseSegmentMean]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.598 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SparseSegmentMeanGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.613 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SparseSegmentSum]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.628 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SparseToDense]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.644 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SpatialTransformer]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.661 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SplitD]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.675 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SplitV]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.691 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Sqrt]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.708 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SqrtGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.723 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Square]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.739 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[SquaredDifference]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.756 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[StatelessBernoulli]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.770 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[StatelessRandomUniformV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.784 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[StatelessRandperm]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.800 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[StridedSlice]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.815 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[StridedSliceV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.832 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Sub]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.851 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TableToResource]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.867 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Tan]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.880 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TanhGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.895 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TensorEqual]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.910 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TensorScatterUpdate]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.926 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ThreeNN]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.940 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Tile]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.955 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TileWithAxis]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.968 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TopK]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.983 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TopKPQDistance]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.987.996 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TopKPQDistanceV2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.011 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TopKV2D]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.024 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Trace]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.046 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TransData]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.062 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[TransDataRNN]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.075 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Tril]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.093 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Triu]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.108 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UniformCandidateSampler]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.124 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UninitEmbeddingHashmap]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.138 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UninitPartitionMap]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.155 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UniqueConsecutive]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.185 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UniqueWithCountsAndSorting]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.201 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UniqueWithCountsExt2]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.218 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UpsampleNearest3d]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.235 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UpsampleNearest3dGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.250 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UpsampleTrilinear3d]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.266 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[UpsampleTrilinear3dGrad]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.286 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ViewCopy]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.302 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Voxelization]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.316 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[WarpAffine]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.331 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[ZerosLike]." [INFO] AICPU(619427,python3.9):2025-04-03-09:35:43.988.346 [folding.cc:413][InitCpuConstantFoldingNew][tid:619427]:"Register op[Zeta]." [INFO] OP(619427,python3.9):2025-04-03-09:35:43.988.369 [host_cpu_kernel_wrapper.cpp:118][OP_PROTO][Initialize][619427] OpName:[HostCpuKernelWrapperOp] "Host cpu init success" [INFO] GE(619427,python3.9):2025-04-03-09:35:43.988.386 [host_cpu_engine.cc:273]619427 LoadLib:Lib: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/libops_host_cpu.so has been opened [INFO] GE(619427,python3.9):2025-04-03-09:35:43.988.543 [plugin_manager.cc:872]619427 LoadWithFlags:Dlopen path: /usr/local/Ascend/ascend-toolkit/8.1.RC1/aarch64-linux/lib64/plugin/engines/runtime/libdvpp_rtkernel.so. flags is 4354 [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.992.144 [plugin_manager.cc:883]619427 LoadWithFlags:The shared library will not be checked. Please ensure that the source of the shared library is trusted. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.233 [ops_kernel_executor_manager.cc:37]619427 Initialize:No library to load [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.252 [plugin_manager.cc:540]619427 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.265 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.309 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.335 [plugin_manager.cc:547]619427 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.992.352 [plugin_manager.cc:206]619427 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.224 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.268 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.286 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.304 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.352 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.699 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.733 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.749 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.764 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.800 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.842 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.860 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.875 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.903 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.939 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.956 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.993.970 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.993.986 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.994.321 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.357 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.373 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.386 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.994.424 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.994.764 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.798 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.816 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.834 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.994.870 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.905 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.921 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.948 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.994.964 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.994.995 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.012 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.024 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.995.040 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.995.373 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.407 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.424 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.436 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.995.479 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.995.816 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.850 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.866 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.880 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.995.914 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.950 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.966 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.995.990 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.003 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.033 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.047 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.059 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.072 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.421 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.456 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.474 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.486 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.519 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.858 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.890 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.907 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.921 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.996.958 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.996.993 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.010 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.035 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.048 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.078 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.092 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.108 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.122 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.456 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.488 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.504 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.519 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.554 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.897 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.930 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.947 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.997.959 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.997.996 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.033 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.050 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.072 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.085 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.115 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.131 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.143 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.160 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.171 [plugin_manager.cc:218]619427 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.193 [plugin_manager.cc:422]619427 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.210 [plugin_manager.cc:155]619427 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.232 [plugin_manager.cc:158]619427 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.247 [plugin_manager.cc:426]619427 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.572 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.607 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.624 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.998.642 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.689 [plugin_manager.cc:439]619427 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619427,python3.9):2025-04-03-09:35:43.998.711 [ge_executor.cc:259]619427 InitOpsProtoManager:Get opsproto path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/minds [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.079 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.112 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.135 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.240 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.259 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.338 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.355 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.999.386 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.435 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.461 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.999.493 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.541 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.559 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.626 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.642 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.710 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.727 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.999.758 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.805 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.821 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:43.999.850 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.894 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.910 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 4 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.975 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:43.999.991 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.056 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.073 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.125 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.180 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.200 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 5 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.228 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.274 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.291 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 6 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.358 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.374 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.440 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.458 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.488 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.531 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.546 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 7 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.574 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.616 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.631 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 8 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.696 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.718 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.781 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.796 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.825 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.868 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.883 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 9 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.910 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.954 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.000.970 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 10 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.000.996 [opsproto_manager.cc:123]619427 GetOpsProtoSoFileList:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] unsuccessful, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.039 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 10 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.106 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.123 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.137 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.152 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 3 libs. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.001.166 [opsproto_manager.cc:147]619427 LoadOpsProtoPluginSo:[LoadSo][Check] Shared library will not be checked. Please make sure that the source of shared library is trusted. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.550 [operator_factory_impl.cc:215]619427 RegisterInferShapeFunc:operator_infershape_funcs_ init [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.574 [operator_factory_impl.cc:269]619427 RegisterVerifyFunc:operator_verify_funcs_ init [INFO] GE(619427,python3.9):2025-04-03-09:35:44.001.611 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.007.866 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Igammac] has registered infer func [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.703 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.847 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.895 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.928 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.960 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.008.991 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.009.022 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.009.051 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto//libop_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.009.081 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto//libcust_op_proto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.029.330 [operator_factory_impl.cc:368]619427 RegisterInferAxisTypeInfoFunc:axis type info derivation funcs init [INFO] GE(619427,python3.9):2025-04-03-09:35:44.029.593 [operator_factory_impl.cc:284]619427 RegisterInferDataSliceFunc:operator_infer_data_slice_funcs_ init [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.029.794 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MatrixBandPart] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.029.821 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [CheckNumerics] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.029.837 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [UnravelIndex] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.029.853 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [UpperBound] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.029.868 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ListDiff] has registered infer func [INFO] GE(619427,python3.9):2025-04-03-09:35:44.029.894 [operator_factory_impl.cc:254]619427 RegisterInferFormatFunc:operator_inferformat_funcs_ init [INFO] GE(619427,python3.9):2025-04-03-09:35:44.029.920 [operator_factory_impl.cc:340]619427 RegisterInferAxisSliceFunc:axis slice derivation funcs init [INFO] GE(619427,python3.9):2025-04-03-09:35:44.029.967 [operator_factory_impl.cc:307]619427 RegisterInferValueRangeFunc:operator_infervalue_paras_ init [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.030.000 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [IdentityN] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.030.023 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [LowerBound] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.030.043 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Expand] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.030.163 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [LeftShift] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.041 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Reciprocal] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.073 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Sign] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.093 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Div] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.115 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Exp] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.138 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [LessEqual] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.156 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Log1p] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.180 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DivNoNan] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.224 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Mul] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.246 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [SqrtGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.266 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Log] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.285 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AddN] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.312 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AcosGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.330 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AcoshGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.364 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [BiasAdd] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.396 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FloorDiv] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.419 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ArgMin] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.488 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MulNoNan] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.532 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AsinGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.554 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Cos] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.572 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Sin] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.601 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AsinhGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.031.637 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Expm1] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.032.950 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AdjustHue] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.032.984 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [AdjustSaturation] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.007 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ExtractGlimpse] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.031 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ResizeArea] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.049 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ResizeBicubic] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.070 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [RGBToHSV] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.090 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [NonMaxSuppressionWithOverlaps] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.131 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ScaleAndTranslate] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.148 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ScaleAndTranslateGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.165 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [CombinedNonMaxSuppression] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.265 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [CholeskyGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.284 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Cholesky] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.300 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MatrixDeterminant] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.316 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MatrixSolve] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.336 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MatrixTriangularSolve] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.351 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Qr] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.682 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Digamma] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.705 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Igamma] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.724 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Igammac] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.742 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Bincount] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.756 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Betainc] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.772 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Bucketize] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.791 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [IgammaGradA] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.819 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [NextAfter] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.850 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [IsInf] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.866 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [IsNan] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.883 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Conj] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.910 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Complex] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.033.926 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Angle] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.173 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ScatterNdUpdate] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.196 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [TensorScatterUpdate] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.257 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Tril] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.275 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [IndexPut] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.292 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Eye] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.309 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FillDiagonal] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.034.325 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Trace] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.264 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DecodeBbox] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.296 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ClipBoxes] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.317 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ClipBoxesD] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.339 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DecodeBoundariesTarget] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.357 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DecodeCornerpointsTargetBG] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.375 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DecodeCornerpointsTargetWrtCenterV1] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.392 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DecodeWheelsTarget] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.523 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [SparseSoftmaxCrossEntropyWithLogits] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.035.615 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [KlDivLossGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.005 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FractionalMaxPoolGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.028 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FractionalAvgPool] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.046 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FractionalMaxPool] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.063 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DataFormatVecPermute] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.090 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [FractionalAvgPoolGrad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.106 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [NthElement] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.613 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Tanh] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.955 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Pad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.982 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [PadV3] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.036.999 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [PadV3Grad] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.020 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [BroadcastTo] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.190 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [RaggedRange] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.346 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ShuffleChannel] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.647 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ReduceSum] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.676 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ReduceProd] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.718 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.037.735 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [INInferV2] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.344 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [UnsortedSegmentSum] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.387 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ReverseV2] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.407 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Select] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.431 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [SegmentSum] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.457 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Slice] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.479 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [OneHot] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.503 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [ScatterNd] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.528 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Cumprod] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.582 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [CumulativeLogsumexp] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.038.624 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [MaskedSelect] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.039.045 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Concat] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.039.244 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [NonDeterministicInts] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.039.531 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [Flatten] has registered infer func [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.039.575 [operator_factory_impl.cc:222]619427 RegisterInferShapeFunc:op [DepthToSpace] has registered infer func [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.752 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.786 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.816 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.851 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.884 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.917 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.967 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.042.996 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.026 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.052 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.078 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.102 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.126 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.151 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.174 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.254 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.276 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.301 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.325 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.350 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.387 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.417 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.442 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.502 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.534 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.582 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.662 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.688 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.711 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.790 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.814 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.841 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.924 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.043.983 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.035 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.057 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.141 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.187 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.213 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.272 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.307 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.332 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.358 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.387 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.414 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.468 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.492 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.540 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.594 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.651 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.677 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.704 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.728 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.787 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.816 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.846 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.930 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.955 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.044.978 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.003 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.035 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.057 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.082 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.144 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.167 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.195 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.219 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.304 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.327 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.352 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.377 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.404 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.429 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.454 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.485 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.538 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.562 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.592 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.616 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.640 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.663 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.687 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.709 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.761 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.786 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.810 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.835 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.861 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.910 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.965 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.045.991 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.041 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.087 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.156 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.183 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.205 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.227 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.253 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.277 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.326 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.351 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.375 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.452 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.477 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.523 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.547 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.569 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.614 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.769 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.824 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.857 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.882 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.906 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.934 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.961 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.046.986 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.012 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.038 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.064 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.093 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.120 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.144 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.168 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.197 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.222 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.246 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.328 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.353 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.380 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.433 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.483 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.537 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.561 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.641 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.670 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.695 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.724 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.748 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.774 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.803 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.829 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.854 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.879 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.903 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.927 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.954 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.047.980 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.005 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.060 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.087 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.111 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.140 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.185 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.218 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.240 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.264 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.312 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.336 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.361 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.385 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.410 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.437 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.482 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.509 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.533 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.558 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.585 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.613 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.635 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.659 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.687 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.711 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.872 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.048.982 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.106 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.217 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.378 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.442 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.466 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.583 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.691 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.803 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.049.922 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.031 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.162 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.310 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.336 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.363 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.471 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.589 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.619 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.644 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.668 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.690 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.719 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.752 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.870 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.050.980 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.110 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.221 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.248 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.368 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.524 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.546 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.649 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.763 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.051.981 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.119 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.238 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.384 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.606 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.751 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.861 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.052.970 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.087 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.194 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.301 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.407 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.444 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.476 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.503 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.525 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.553 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.578 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.602 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.627 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.659 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.684 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.709 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.817 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.930 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.958 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.053.981 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.004 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.037 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.058 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.179 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.401 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.532 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.641 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [InferShape][InferShapeRange][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.669 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.806 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.054.916 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.146 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.257 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.369 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.806 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.922 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.055.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.104 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.228 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.259 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.363 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.584 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.690 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.802 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.056.944 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.057 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.166 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.405 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.627 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.733 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.838 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.950 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.057.980 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.005 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.131 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.158 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.181 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.204 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.250 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.270 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.291 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.312 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.332 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.353 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.374 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.396 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.416 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.439 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.461 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.481 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.589 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.701 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.808 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.058.911 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.259 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.291 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.318 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.340 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.444 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.470 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.495 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.630 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.652 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.674 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.696 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.800 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.059.913 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.020 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.130 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.255 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.362 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.465 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.606 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.831 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.060.931 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.051 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.163 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.221 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.245 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.348 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.456 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.572 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.680 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.785 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.061.898 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.003 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.116 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.237 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.353 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.580 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.722 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.835 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.959 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.062.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.013 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.041 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.073 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.099 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.143 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.392 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.607 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.717 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.829 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.063.949 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.061 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.086 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.203 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.320 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.546 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.661 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.767 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.888 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.917 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.943 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.064.977 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.004 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.028 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.078 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.102 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.126 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.150 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.181 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.206 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.255 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.305 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.329 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.354 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.576 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.607 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.720 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.831 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.065.943 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.020 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.130 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.362 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.447 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.559 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.664 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.776 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.880 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.066.995 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.114 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.346 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.371 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.479 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.588 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.700 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.815 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.926 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.067.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.057 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.295 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.644 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.765 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.068.990 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.124 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.234 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.345 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.592 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.706 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.816 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.069.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.055 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.175 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.296 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.408 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.515 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.627 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.842 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.867 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.070.992 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.105 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.218 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.323 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.433 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.750 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.864 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.071.974 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.080 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.207 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.330 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.551 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.653 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.072.900 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.012 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.119 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.145 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.164 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.185 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.209 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.239 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.262 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.283 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.304 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.431 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.453 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.554 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.665 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.773 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.881 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.073.992 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.018 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.040 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.143 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.250 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.383 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.486 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.605 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.712 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.735 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.767 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.792 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.904 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.930 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.074.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.088 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.111 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.218 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.245 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.268 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.290 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.503 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.609 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.633 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.656 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.765 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.791 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.075.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.007 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.146 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.176 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.287 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.312 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.338 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.554 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.664 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.771 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.880 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.076.985 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.093 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.204 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.321 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.430 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.542 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.647 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.873 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.077.985 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.094 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.145 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.263 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.287 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.309 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.416 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.525 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.629 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.742 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.850 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.078.959 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.077 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.184 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.300 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.415 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.441 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.463 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.739 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.860 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.079.973 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.079 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.200 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.313 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.543 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.646 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.750 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.861 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.080.972 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.080 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.299 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.413 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.747 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.858 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.081.965 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.074 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.188 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.294 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.404 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.660 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.082.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.114 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.232 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.355 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.468 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.580 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.690 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.802 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.083.917 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.147 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.267 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.374 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.482 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.589 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.712 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.821 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.084.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.047 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.155 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.499 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.714 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.085.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.070 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.219 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.256 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.293 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.336 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.458 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.487 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.514 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.539 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.563 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.587 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.609 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.645 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [InferShape][InferDataType][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.669 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.778 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.086.893 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.004 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.227 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.356 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.464 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.593 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.708 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.817 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.087.925 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.048 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.154 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.311 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.334 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.378 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.565 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.598 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.668 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.721 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.796 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.857 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.886 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.918 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.088.979 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.072 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [InferShape][InferDataType][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.101 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [InferShape][InputsDependency][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.311 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.343 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.453 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.565 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.679 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.089.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.010 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.047 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.159 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.380 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.488 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.513 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.622 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.734 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.838 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.090.950 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.029 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.110 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.221 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.334 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.447 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [InferShape][InferShapeRange][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.678 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.091.961 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.082 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.205 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.317 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.434 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [InferShape][IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.547 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.578 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.683 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.796 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.092.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.012 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.129 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.397 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.509 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.729 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.842 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.093.995 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [IsPrivateAtrrReg] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.031 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.060 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.086 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.114 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.171 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.277 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.411 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.544 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.571 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.674 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.781 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.094.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.009 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.114 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.225 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.255 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.452 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.532 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.675 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.783 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.095.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.364 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.476 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.562 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.600 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.704 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.815 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.096.932 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.150 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.259 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.458 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.487 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.715 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.097.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.076 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.188 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [InferShape][InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.306 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.340 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.367 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.418 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.444 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.470 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.574 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.598 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.650 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.098.769 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [InferShapeRange][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.000 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.119 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.151 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.178 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.395 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.505 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.618 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.737 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.763 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.787 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.808 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.832 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.099.938 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.169 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.292 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.324 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.365 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [InferShape][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.485 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.514 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.625 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [InferShapeRange] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.735 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.845 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.100.952 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.064 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.091 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.208 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.339 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.366 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.421 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.447 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.523 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.547 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.574 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.599 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.625 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.684 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.711 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.820 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.846 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.870 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.101.998 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.027 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.055 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.108 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.133 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.158 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.184 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.209 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.102.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [InferShape][InferDataType][InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.394 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.577 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [InferShape] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.659 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.824 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.103.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.073 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.193 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.442 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.467 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InferShape][InferDataType] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.611 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.694 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.912 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.104.994 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.105.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.105.432 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [InputsDependency] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.105.461 [opsproto_manager.cc:162]619427 LoadOpsProtoPluginSo:OpsProtoManager plugin load /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto.so success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.105.509 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.105.552 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.014 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.065 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.089 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.169 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.186 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.247 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.276 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.290 [opp_so_manager.cc:56]619427 LoadOpsProtoPackage:Start load ops proto package, is_split:[1]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.307 [plugin_manager.cc:540]619427 GetOpsProtoPath:Enter GetOpsProtoPath schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.322 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.345 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.363 [plugin_manager.cc:547]619427 GetOpsProtoPath:Opp plugin path structure is new version! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.382 [plugin_manager.cc:206]619427 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.789 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.106.829 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.106.847 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.106.860 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.106.904 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.256 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.288 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.306 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.322 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.357 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.404 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.423 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.440 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.457 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.489 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.505 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.518 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.534 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.869 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.903 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.921 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.107.935 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.107.973 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.108.327 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.365 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.382 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.395 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.108.429 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.476 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.492 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.506 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.108.520 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.550 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.569 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.584 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.108.598 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.108.938 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.973 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.108.991 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.009 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.109.053 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.109.408 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.440 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.459 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.474 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.109.510 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.564 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.583 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.598 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.109.615 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.651 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.667 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.109.680 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.109.695 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.043 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.077 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.092 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.104 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.136 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.484 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.518 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.535 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.547 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.593 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.637 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.654 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.668 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.681 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.713 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.731 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.110.749 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.110.766 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.114 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.146 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.163 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.181 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.216 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.578 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.612 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.628 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.641 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.687 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.734 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.750 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.764 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.778 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.808 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.824 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.111.839 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.853 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.868 [plugin_manager.cc:218]619427 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/ [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.888 [plugin_manager.cc:422]619427 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.903 [plugin_manager.cc:155]619427 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.925 [plugin_manager.cc:158]619427 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.111.940 [plugin_manager.cc:426]619427 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.307 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.112.346 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.112.364 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.112.379 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.418 [plugin_manager.cc:439]619427 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspo [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.742 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.774 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.796 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.936 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.953 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.112.968 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.115.608 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.115.644 [op_impl_registry_holder_manager.cc:216]619427 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.115.663 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.115.730 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.116.292 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.116.426 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.116.444 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.116.458 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.118.685 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.118.719 [op_impl_registry_holder_manager.cc:216]619427 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.118.738 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.118.779 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.186 [op_impl_space_registry.cc:80]619427 MergeFunctions:op type DecoderKvCache infer_shape func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.210 [op_impl_space_registry.cc:83]619427 MergeFunctions:op type DecoderKvCache infer_datatype func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.231 [op_impl_space_registry.cc:80]619427 MergeFunctions:op type PromptKvCache infer_shape func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.243 [op_impl_space_registry.cc:83]619427 MergeFunctions:op type PromptKvCache infer_datatype func has been registered [INFO] GE(619427,python3.9):2025-04-03-09:35:44.119.258 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so] success! [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.303 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.119.341 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.119.432 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.119.451 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.119.467 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.120.245 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.120.284 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619427,python3.9):2025-04-03-09:35:44.120.378 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.120.397 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.120.409 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.057 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.121.079 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.121.121 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.121.154 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.237 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.254 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.266 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.875 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.121.899 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619427,python3.9):2025-04-03-09:35:44.121.991 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.007 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.019 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.613 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.122.642 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.122.680 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.122.714 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.796 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.813 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.122.826 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.123.412 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.123.433 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619427,python3.9):2025-04-03-09:35:44.123.522 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.123.538 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.123.550 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.124.133 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.124.154 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.124.210 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.124.244 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.124.338 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.124.355 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.124.367 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.124.948 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.124.969 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.067 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.086 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.101 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.680 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.125.702 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_proto/lib/linux/aarch64//libcust_opsproto_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.125.745 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.125.781 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.125.821 [opp_so_manager.cc:97]619427 LoadOpsProtoSo:[FindSo][Check] Get path with os&cpu type [/usr/local/Ascend/ascend-toolkit/latest/opp/op_proto/custom/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.892 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 0 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.939 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.956 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.125.969 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.706 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EluGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.783 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.807 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.826 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Abs] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.865 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Real] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.886 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Acos] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.905 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Acosh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.924 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddMatMatElements] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.945 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Adds] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.963 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Asin] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.151.985 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Asinh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.005 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AssignAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.025 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AssignSub] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atan] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.064 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atanh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.100 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BesselI0e] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.119 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BesselI1e] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.167 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.188 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.207 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.227 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bucketize] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cast] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.270 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ceil] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.292 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CeluV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.322 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ComplexAbs] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.340 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cos] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cosh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DataFormatDimMap] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.414 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dequantize] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.431 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Elu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.450 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erf] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.470 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erfc] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Exp] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.512 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expm1] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantWithMinMaxVars] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.548 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGelu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.564 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGeluV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.585 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Floor] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.604 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulAddN] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSigmoid] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.641 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardShrink] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardShrinkGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.675 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSwish] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.693 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSwishGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.710 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardtanhGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.726 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexFillD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.745 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.761 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.783 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Inv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.810 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Invert] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.830 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Log] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.849 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Log1p] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.866 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalNot] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.885 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogAddExp] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.904 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.923 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormUpdateV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dawsn] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.961 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expint] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.152.983 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Erfinv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.002 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FresnelCos] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.021 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FresnelSin] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.040 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ndtri] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.059 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Spence] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.078 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fills] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LeakyRelu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.118 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OnesLike] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.137 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Reciprocal] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.155 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sign] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.174 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sqrt] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tanh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.211 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ZerosLike] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gelu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.251 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.270 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Neg] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.307 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSoftmaxV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.333 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.353 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.410 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Muls] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Power] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.444 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.463 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Relu6Grad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.502 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rint] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.539 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Round] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rsqrt] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.576 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RsqrtGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.596 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Selu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.614 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Shrink] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.632 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sigmoid] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.653 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.673 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.693 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sin] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.712 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sinh] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.729 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1Loss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSigmoid] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.787 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSigmoidGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.806 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSpaceD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.825 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Softplus] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.850 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Softsign] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.867 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqrtGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Square] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.907 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Swish] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.926 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwishGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.949 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SiluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.972 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardElemt] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.153.994 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tan] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftShrink] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.038 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftShrinkGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.061 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Threshold] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.104 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.124 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trunc] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.144 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.164 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.187 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MishGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.211 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HardSigmoidGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.252 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.271 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.291 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.318 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FillDiagonal] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.342 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPut] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.362 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReadVariableOp] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.380 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CastLike] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.411 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedBiasLeakyRelu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.432 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessBernoulliV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.453 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.474 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InvertPermutation] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.495 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.513 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SearchSorted] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.535 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.553 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.575 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.596 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.621 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.726 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScale] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.751 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AbsGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.771 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AccumulateNV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.787 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddN] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.805 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eltwise] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.825 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.845 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AcosGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.867 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AcoshGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.889 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsinGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.911 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsinhGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.932 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Atan2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.954 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AtanGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.972 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Axpy] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.154.991 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseAnd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.011 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseOr] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.041 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BitwiseXor] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Div] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.082 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DivNoNan] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.102 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.123 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FastGeluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.145 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FloorDiv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.169 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FloorMod] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.195 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InvGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.216 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LeakyReluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.237 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TanhGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.259 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Add] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Addcdiv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.301 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Addcmul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.322 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.343 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalAnd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.364 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogicalOr] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.384 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.405 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pow] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.426 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RealDiv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.446 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByNormNoDivSum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.469 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByValue] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.493 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ClipByValueV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.537 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sub] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.578 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gcd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.598 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LarsV2Update] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.628 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedFill] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.650 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.672 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xlog1py] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApproximateEqual] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.713 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Equal] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.731 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NotEqual] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.756 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Greater] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.777 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GreaterEqual] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.798 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Less] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.821 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LessEqual] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.840 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Maximum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.865 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Minimum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.886 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mod] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.907 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.929 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulNoNan] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.950 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReciprocalGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.969 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SeluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.155.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.010 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftplusV2Grad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftsignGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.075 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquaredDifference] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.097 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.119 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThresholdGradV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.141 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TruncateDiv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TruncateMod] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.209 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xdivy] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.235 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsClose] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.258 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Xlogy] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.301 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SelectV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.323 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelectV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Lerp] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.365 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedBiasLeakyReluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.471 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.573 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.680 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.782 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.921 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.946 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.156.970 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.068 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.176 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.371 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.474 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.583 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.687 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Quantize] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.712 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendDequant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.758 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.857 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.965 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.157.993 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.036 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.056 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.077 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BasicLSTMInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BasicGRUInplaceFillWindowCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.197 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.299 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.405 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.506 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.629 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaCast] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.727 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.754 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.778 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.801 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.158.895 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.003 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.103 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.202 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.226 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.321 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.417 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.551 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.653 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.755 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.886 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.159.980 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L2Loss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.082 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.195 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.298 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.399 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.494 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.519 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceAny] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceAll] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.563 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.584 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCast] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.631 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.652 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceProd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.671 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceSum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EuclideanNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.711 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceLogSum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.733 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceLogSumExp] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.832 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceNansum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.961 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.160.981 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.019 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.064 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.164 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.267 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.362 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.389 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.488 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.586 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.611 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.633 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.733 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.833 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.161.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.038 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.346 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.547 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.643 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.750 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.774 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.162.994 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.032 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.129 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.223 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.325 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.429 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.534 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.657 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.758 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.854 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.951 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.163.976 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.075 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.190 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.292 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.390 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.490 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.643 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.742 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.769 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.791 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.815 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.837 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.890 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.911 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.932 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.954 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.972 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.164.993 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.015 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.037 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.060 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.082 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.106 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.206 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SegmentSum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.405 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.507 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.612 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.709 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.811 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.836 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentMin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentProd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.884 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.165.979 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOne] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.004 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneAssign] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.026 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecay] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.048 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdamApplyOneWithDecayAssign] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.154 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduce] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.176 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.196 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.217 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.313 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduce] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.421 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.719 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.816 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.166.917 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.028 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.125 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.222 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.319 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.414 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.517 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateGradGammaBeta] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.591 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingUpdateV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.683 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.885 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.167.984 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.091 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.203 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.314 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.415 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.514 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.611 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.721 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.825 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.926 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaximumGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.168.950 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MinimumGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.142 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogSoftmaxGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.167 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.191 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.217 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.241 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.262 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.285 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.308 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.406 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.435 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Mish] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.528 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.628 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.723 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.830 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.169.926 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.033 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxCrossEntropyWithLogits] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.140 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.167 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.256 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.452 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.553 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.655 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.751 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.846 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.875 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.898 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.919 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdaMaxD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.939 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.960 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.170.980 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWithAmsgradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.001 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAddSignD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.023 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCenteredRMSPropD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFtrlD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.065 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.094 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyGradientDescent] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyKerasMomentumD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.135 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyMomentumD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.156 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyPowerSignD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.174 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.195 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyProximalGradientDescent] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.219 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdagradDAD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.347 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.450 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulApplyMomentum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.474 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedMulApplyKerasMomentum] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.572 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.669 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.844 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.171.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNegInf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.043 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsPosInf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.148 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.234 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.331 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.435 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.539 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.741 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReluV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.838 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.172.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.034 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummax] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.158 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.263 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.361 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.462 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2ImV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.559 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.593 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.697 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.793 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.891 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.173.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.081 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.184 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.285 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.388 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.485 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.580 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.682 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.782 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelect] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.880 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.174.907 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.007 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.103 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.199 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.409 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.508 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.708 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.805 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.175.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RNNTLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.017 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultinomialWithReplacement] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.146 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.256 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessBernoulli] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.358 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.459 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.558 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueConsecutive] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.761 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueWithCountsAndSorting] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.855 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpAffine] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.176.957 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Histogram] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.055 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.150 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.252 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unique] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.350 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UniqueWithCounts] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.448 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dot] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.550 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.650 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.748 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.778 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.879 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormBackwardReduce] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.177.977 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStatsWithCounts] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.076 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.100 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.122 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.146 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.178 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.200 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.222 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.242 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.265 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.384 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.409 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.701 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.799 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.895 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.919 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.178.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.037 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.164 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.265 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.366 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.464 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.586 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.612 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.750 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.775 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.921 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.179.943 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.036 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.063 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.084 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.106 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.221 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.328 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.428 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.453 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.476 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.569 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.594 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.693 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.792 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.895 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.920 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.180.945 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.066 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.088 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.195 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.293 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.497 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.595 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.693 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.790 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.181.995 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool2dAssistMatrix] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.194 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.292 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.396 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.503 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cdist] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.600 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CdistGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.730 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.753 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.870 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.182.990 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.194 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.294 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GlobalLpPool] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.391 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GLU] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GLUGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.598 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.698 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.921 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.947 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.183.968 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MinAreaPolygons] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.185 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.288 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.388 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.489 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.593 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.691 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.786 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.888 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.184.986 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.087 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.193 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.291 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.391 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.488 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.587 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.681 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.793 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.185.996 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.093 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.190 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.294 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.388 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.490 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.590 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.614 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.710 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedNMS] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.814 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.186.919 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.015 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheck] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.117 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.310 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3d] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.419 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.518 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActiveRotatedFilter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.720 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActiveRotatedFilterGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.818 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.187.915 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.015 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.229 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.342 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.449 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.551 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSAMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.650 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSAMaskGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.752 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.853 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.188.951 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.053 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.148 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.250 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeNN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.354 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Voxelization] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.469 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.500 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.529 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.583 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.631 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.660 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.684 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.706 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.727 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.750 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.773 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.796 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CompressFcOp] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.189.927 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.032 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.134 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.236 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.337 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.434 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV4] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.532 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.618 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.646 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.190.949 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.048 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeTrilinear] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.147 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.249 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.298 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.317 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.337 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.460 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.490 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.518 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.545 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.569 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.607 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.633 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.689 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.713 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.738 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.764 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.790 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.818 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.849 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.876 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.925 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.191.999 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.100 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActULQClampMaxGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.125 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActULQClampMinGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.234 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActsULQ] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.339 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ActsULQInputGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.441 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.538 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.639 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPoolAssistMatrix] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.737 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.764 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.864 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.192.979 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.083 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.185 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.211 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.306 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.402 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.505 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.608 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.683 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.761 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.193.968 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.075 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.180 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bitcast] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.277 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.355 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.437 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.619 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLoss] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.720 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.822 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.194.909 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.013 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutGenMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.046 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessDropOutGenMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.140 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.241 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.455 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.555 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExpandDims] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.756 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FillWindowCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.195.960 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlattenV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherShapes] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GaussianBlur] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FileConstant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.318 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessIf] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.343 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[If] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.368 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[_If] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.390 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Case] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.434 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConstPlaceHolder] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.533 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.634 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.713 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Identity] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.738 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MemcpyAsync] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IdentityN] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.860 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.196.970 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.071 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.173 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.275 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.371 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.408 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.432 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.576 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.602 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.682 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.782 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.878 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.197.981 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.080 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.179 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.387 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RandomChoiceWithMask] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RandomUniform] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rank] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.582 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Size] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.679 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.786 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Reshape] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.891 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RgbToGrayscale] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.198.992 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.095 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.296 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rotate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.376 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Shape] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ShapeN] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheckV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.614 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SilentCheckV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.714 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.814 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.922 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdadelta] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.947 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdadeltaD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.199.977 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.000 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.022 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.044 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyProximalAdagradD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.065 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.089 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyAdagradV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.112 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.135 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.173 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.198 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.219 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyRMSProp] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.241 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyRMSPropD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.342 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseFillEmptyRows] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.442 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.543 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Squeeze] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.571 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqueezeV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.594 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SqueezeV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.696 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.793 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StatelessRandomNormalV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.892 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.200.993 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.106 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopK] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.132 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.157 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.180 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.205 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.303 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKPQDistanceV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.396 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.494 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransArgb] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.597 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unsqueeze] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsqueezeV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.646 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsqueezeV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.740 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Where] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.201.968 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WtsARQ] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.067 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.169 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.271 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.298 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.401 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.525 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustContrast] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.549 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustHue] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.572 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustSaturation] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.594 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CropAndResize] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.626 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CropAndResizeV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.648 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DecodeAndCropJpeg] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.669 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DecodeJpeg] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NormalizeV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustContrastWithMean] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.739 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustBrightness] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.762 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustBrightnessV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.783 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdjustSaturationV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.806 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpAffineV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgCrop] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.850 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WarpPerspective] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.944 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.970 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.202.991 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.089 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllGather] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.137 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomBroadcast] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.159 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReduceScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.179 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomSend] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.222 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomReceive] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.243 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllToAllV] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.266 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomAllToAllVC] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.288 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HcomGatherAllToAllV] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.414 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.628 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.709 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.783 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.203.856 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.279 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.356 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.435 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.515 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.588 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.658 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.777 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.881 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.204.988 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.205.063 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.205.142 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.205.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.205.236 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.205.499 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.214.395 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 894 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.219.781 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.242.674 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so] success! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.031 [plugin_manager.cc:554]619427 GetUpgradedOpsProtoPath:Start to get upgraded ops proto path [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.054 [plugin_manager.cc:134]619427 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.243.106 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.243.140 [plugin_manager.cc:145]619427 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.243.155 [plugin_manager.cc:557]619427 GetUpgradedOpsProtoPath:Failed to get upgraded opp path! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.169 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.197 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.914 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.971 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:44.243.995 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.080 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_proto/lib/linux/aarch64//libopsproto_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.099 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.181 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.199 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.216 [opp_so_manager.cc:66]619427 LoadOpMasterPackage:Start load op master package, is_split:[1]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.231 [plugin_manager.cc:616]619427 GetOpTilingForwardOrderPath:Enter GetOpTilingPath schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.246 [plugin_manager.cc:108]619427 GetOppPath:Enter get opp path schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.271 [plugin_manager.cc:117]619427 GetOppPath:Get opp path from env: /usr/local/Ascend/ascend-toolkit/latest/opp [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.292 [plugin_manager.cc:624]619427 GetOpTilingForwardOrderPath:Opp plugin path structure is new version! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.311 [plugin_manager.cc:206]619427 GetPluginPathFromCustomOppPath:value of env custom_op_lib_path_ is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops:/ [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.690 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.244.731 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.244.763 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.244.777 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.244.822 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.173 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.213 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.231 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.246 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.282 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.329 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.348 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.363 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.378 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.395 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.735 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.769 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.786 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.245.802 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.245.842 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.199 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.235 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.252 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.264 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.300 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.339 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.356 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.369 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.383 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.397 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.740 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.776 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.793 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.246.811 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.246.849 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.188 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.221 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.238 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.251 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.297 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.342 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.360 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.374 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.389 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.405 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.753 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.788 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.806 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.247.823 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.247.858 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.227 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.266 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.286 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.300 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.335 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.378 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.394 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.423 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.439 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.453 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.802 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.838 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.853 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.248.866 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.248.905 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.260 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.295 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.313 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.328 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.365 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b' is valid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.409 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.426 [plugin_manager.cc:1026]619427 GetVersionFromPathWithName:Invalid input file path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/version.info], make sure that the file path is correct. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.440 [plugin_manager.cc:322]619427 IsVendorVersionValid:[NotVerification] Will not verify version as the opp version and compiler version are not set [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.456 [plugin_manager.cc:212]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops' is valid. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.491 [plugin_manager.cc:215]619427 GetPluginPathFromCustomOppPath:custom_path '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicpu_ops' is invalid, which is skipped. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.505 [plugin_manager.cc:218]619427 GetPluginPathFromCustomOppPath:Run GetPluginPathFromCustomOppPath finished, current plugin_path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-pack [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.525 [plugin_manager.cc:422]619427 GetOppPluginPathNew:Enter get opp plugin path new schedule [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.542 [plugin_manager.cc:155]619427 GetOppPluginVendors:Enter get opp plugin config file schedule, config file is '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini' [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.569 [plugin_manager.cc:158]619427 GetOppPluginVendors:Can not open file '/usr/local/Ascend/ascend-toolkit/latest/opp/vendors/config.ini'! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.584 [plugin_manager.cc:426]619427 GetOppPluginPathNew:Can not get opp plugin vendors! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.249.926 [plugin_manager.cc:250]619427 GetRequiredOppAbiVersion:extract required opp abi version info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/compiler//version.info [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.963 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [Version=7.7.T20.0.B210], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.980 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [version_dir=8.1.RC1], it must include [required_opp_abi_version=]. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.249.994 [plugin_manager.cc:1063]619427 ParseVersion:Incorrect line [timestamp=20250402_001522762], it must include [required_opp_abi_version=]. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.033 [plugin_manager.cc:439]619427 GetOppPluginPathNew:plugin_path is '/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_ [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.363 [plugin_manager.cc:996]619427 GetCurEnvPackageOsAndCpuType:extract os and cpu info from /usr/local/Ascend/ascend-toolkit/8.1.RC1/opp/scene.info [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.393 [plugin_manager.cc:1011]619427 GetCurEnvPackageOsAndCpuType:Get os:linux [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.425 [plugin_manager.cc:1015]619427 GetCurEnvPackageOsAndCpuType:Get cpu:aarch64 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.250.484 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.591 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.610 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.250.625 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.315 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff151622c0, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.357 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.381 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.401 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.425 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.747 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff15166978, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.772 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.791 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.810 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.252.829 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.105 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.253.134 [op_impl_registry_holder_manager.cc:216]619427 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.154 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.222 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.498 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.253.558 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.668 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.687 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.253.699 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.163 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:AllFinite, funcPointer:(nil), registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.195 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:AllFinite, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.213 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.234 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.252 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:AllFinite, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.534 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:DecoderKvCache, funcPointer:0xffff151139b0, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.560 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DecoderKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.582 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.603 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.619 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:DecoderKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.904 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:PromptKvCache, funcPointer:0xffff15116da0, registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.927 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:PromptKvCache, funcPointer:(nil), registered count:6 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.944 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.974 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.255.990 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:PromptKvCache, funcPointer:(nil), registered count:3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.256.273 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.305 [op_impl_registry_holder_manager.cc:216]619427 GetOpImplFunctionsByHandle:Get op impl functions failed, path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so, errmsg:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so: undefined symbol: GetOpImplFunctionsV2 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.256.324 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 3 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.256.362 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.726 [op_impl_space_registry.cc:84]619427 MergeFunctions:op type DecoderKvCache tiling_parse func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.746 [op_impl_space_registry.cc:85]619427 MergeFunctions:op type DecoderKvCache compile_info_creator func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.761 [op_impl_space_registry.cc:86]619427 MergeFunctions:op type DecoderKvCache compile_info_deleter func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.778 [op_impl_space_registry.cc:87]619427 MergeFunctions:op type DecoderKvCache tiling func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.790 [op_impl_space_registry.cc:93]619427 MergeFunctions:op type DecoderKvCache max_tiling_data_size has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.810 [op_impl_space_registry.cc:84]619427 MergeFunctions:op type PromptKvCache tiling_parse func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.823 [op_impl_space_registry.cc:85]619427 MergeFunctions:op type PromptKvCache compile_info_creator func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.839 [op_impl_space_registry.cc:86]619427 MergeFunctions:op type PromptKvCache compile_info_deleter func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.851 [op_impl_space_registry.cc:87]619427 MergeFunctions:op type PromptKvCache tiling func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.866 [op_impl_space_registry.cc:93]619427 MergeFunctions:op type PromptKvCache max_tiling_data_size has been registered [INFO] GE(619427,python3.9):2025-04-03-09:35:44.256.881 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so] success! [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.937 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.256.978 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.257.028 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.128 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.147 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.159 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.727 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.257.750 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.257.796 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.893 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.911 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.257.924 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.258.582 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.258.604 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.258.648 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.258.685 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.258.720 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.258.827 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.258.846 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.258.860 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.259.299 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.259.322 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.259.368 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.259.465 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.259.482 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.259.496 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.085 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.105 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.145 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.192 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.232 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.336 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.366 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.381 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.832 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.853 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.260.897 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.260.991 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.008 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.023 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.611 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.261.633 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.261.682 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.261.720 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.261.756 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.848 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.874 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.261.888 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.262.325 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.262.345 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.262.392 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.262.485 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.262.502 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.262.514 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.102 [op_impl_space_registry.cc:240]619427 ConvertSoToRegistry:So already loaded! so path:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.123 [opp_so_manager.cc:48]619427 LoadSoAndInitDefault:Save so failed, path is /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_ascendc_910b/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libcust_opmaster_rt2.0.so [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.167 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.203 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend/custom_aicore_ops/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.242 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.269 [opp_so_manager.cc:126]619427 LoadOpMasterSo:Get path with op_tiling path [/usr/local/Ascend/ascend-toolkit/latest/opp/op_impl/custom/ai_core/tbe/op_tiling/lib/linux/aarch64/] failed, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.263.303 [opp_so_manager.cc:123]619427 LoadOpMasterSo:Get path with op_master path [/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_master/lib/linux/aarch64/] failed, reason:No such file or directory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.388 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.414 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 1 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.461 [plugin_manager.cc:1126]619427 GetFileListWithSuffix:PluginManager Parse full name = /usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.478 [plugin_manager.cc:1130]619427 GetFileListWithSuffix:Found 2 libs. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.263.492 [opp_so_manager.cc:45]619427 LoadSoAndInitDefault:Start to LoadSoAndInitDefault, opp version:0 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.276.358 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.277.263 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.277.319 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 1 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.277.349 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 11 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.279.657 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_ct.so] success! [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.292.206 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.292.249 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.292.254 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.270 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.292.279 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.288 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.298 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.314 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.292.327 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.292.328 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.292.348 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.292.354 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.384 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.292.384 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.380.781 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DefaultImpl] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.254 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.415 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.492 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuantV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.560 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.636 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.660 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.725 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Concat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.748 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.769 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.907 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.381.975 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.045 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.118 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexByTensor] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.185 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.260 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.349 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.392 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.463 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.532 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.599 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.671 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.755 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.819 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.888 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.382.959 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.029 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.168 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.237 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.306 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.376 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.451 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.522 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.383.590 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmulV3] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.028 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyRotaryPosEmb] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.173 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchToSpaceND] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.571 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cumsum] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.691 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsets] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.801 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableOffsetsGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.391.910 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DenseBincount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.089 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.220 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropOutDoMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.377 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.524 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantUpdateScatterV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.636 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractGlimpseV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Flatten] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.392.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNonFiniteCheckAndUnscale] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.012 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherNd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.126 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.155 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Gather] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.316 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherV3] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.425 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenerateBoundingBoxProposals] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.570 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSilu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.689 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramFixedWidth] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransform] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.824 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImageProjectiveTransformV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.393.941 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsNan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.047 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinSpace] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.314 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MemSet] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.419 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MirrorPad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.544 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.673 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonFiniteCheck] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.782 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonZero] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.394.892 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NonMaxSuppressionV6] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.000 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NPUClearFloatStatus] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.110 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NPUGetFloatStatus] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.260 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.469 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantUpdateScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.601 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RaggedBinCount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.395.758 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.054 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:ReverseSequence, funcPointer:0xfffee716e360, registered count:5 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.087 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:ReverseSequence, funcPointer:(nil), registered count:7 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.113 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_support_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.133 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:get_op_specific_info, op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.170 [op_check.cc:71]619427 RegisterParamGeneralize:RegisterParamGeneralize: op_type:ReverseSequence, funcPointer:(nil), registered count:4 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.196 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseSequence] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.335 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.449 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.653 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMaxWithArgmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.814 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.925 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.396.952 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.081 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterDiv] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.189 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.308 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.334 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdSub] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.442 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNdUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.549 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterNonAliasingAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.657 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterSub] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.771 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.969 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Slice] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.397.997 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SliceWithAxes] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.112 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatch] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.221 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToBatchND] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.418 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseBincount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.534 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseSegmentSumGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.652 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.760 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TensorMove] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.398.868 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UnsortedSegmentSum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.088 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.253 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.398 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.546 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyCamePart4] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.675 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendAntiQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.790 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.399.906 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendFaceBgPartOne] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.013 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUCellGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.127 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicGRUV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.418 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicLSTMGradCell] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.595 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.624 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNN, funcPointer:0xfffee71f8e08, registered count:7 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.797 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.821 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:DynamicRNNV2, funcPointer:0xfffee7201298, registered count:7 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.400.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicRNNV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.054 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GenADC] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.219 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSample] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.453 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandle] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.672 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ImgRawDecodePostHandleV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.788 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LUT3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.401.899 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PasteSubImg] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.011 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.039 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdEnvMatACalcDescrpt] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.152 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdForceSeA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.259 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ProdVirialSeA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.370 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RGB2YUV422] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.475 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanPQCodes] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.587 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScanSQCodes] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.700 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusion] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.811 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TabulateFusionGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.402.923 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YUV4442YUV422] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.074 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.102 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.127 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMaxWithValue] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.151 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ArgMinWithValue] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.313 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AscendQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.428 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Assign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AsStrided] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.656 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropy] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.770 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CheckValid] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.877 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.912 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatV2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.403.942 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pack] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.050 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ConcatOffset] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.172 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Diag] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.286 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagPart] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.481 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicAtomicAddrClean] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.589 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDiv] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.702 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KLDivV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.823 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.404.935 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Range] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.153 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMean] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.180 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.288 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.427 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.713 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.833 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeNearestNeighborV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.405.943 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReverseV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.062 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.170 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ROIAlignGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.287 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseReshape] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.399 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Split] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SplitV] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.723 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumV1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.837 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSlice] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.406.974 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransData] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.409.828 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthToSpace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.409.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Transpose] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.409.912 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SpaceToDepth] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.053 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Unpack] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.175 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Bias] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.285 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.407 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BiasAddGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.520 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BinaryCrossEntropyGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.628 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferenceD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.743 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInfer] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.856 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNInferGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.410.990 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.021 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.132 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.158 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.300 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BN3DTrainingUpdateGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.407 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.516 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BNTrainingUpdateV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BroadcastTo] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.753 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Fill] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.411.989 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantSwigluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.412.304 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.412.450 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeGluV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.412.583 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[INTrainingReduceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.412.700 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.412.815 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormBetaGammaBackpropV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.413.488 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormGradV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.414.311 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.414.641 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormV4] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.415.105 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchNormV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.415.716 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PostLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.415.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.415.966 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormXBackpropV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.416.110 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LayerNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.416.247 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.416.534 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3DWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.418.768 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPool3DGradWithArgmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.419.613 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopK] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.419.782 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MSELossV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.419.948 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV3Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.420.972 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV3] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.421.136 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pows] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.421.300 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3DGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.421.772 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PRelu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.421.892 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PReluGradUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.014 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Scale] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Select] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.248 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidCrossEntropyWithLogitsGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.360 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tile] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.494 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.603 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TileWithAxis] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.720 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[OneHot] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.836 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.422.951 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3GradReplication] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.226 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pdist] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.345 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.538 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElementsV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.649 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftMarginLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.765 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RnnGenMaskV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.423.884 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cummin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.038 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AngleV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.227 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CoalesceSparse] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.345 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Col2im] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.455 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.483 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CTCLossV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.679 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingBag] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.792 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.424.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Eye] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.090 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FakeQuantAffineCachemask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.379 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.490 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.602 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Index] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.710 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.748 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolGradWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.870 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.425.898 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaxPoolWithArgmaxV1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.004 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultilabelMarginLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.115 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sort] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.142 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.253 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StrideAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.359 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.477 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncBatchNormGatherStats] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.664 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[HistogramV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.780 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GatherElements] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.426.966 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler2DGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.124 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GridSampler3DGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GetPaddingOffset] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.479 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadInfer] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.657 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV4Grad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.849 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PadV3GradReplicate] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.427.964 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.074 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Roll] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.192 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Trace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.311 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Tril] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.338 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Triu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.529 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopPSample] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.645 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KlDivLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.753 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[L1LossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.428.982 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.429.095 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceMeanWithCount] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.429.212 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdWithMean] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.429.237 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReduceStdV2Update] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.429.346 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SmoothL1LossV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.429.576 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[EmbeddingDenseGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.430.643 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.433.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeGatingTopKSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.433.578 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.433.784 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.434.121 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRouting] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.434.284 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.434.443 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.263 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[STFT] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.432 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeMerge] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.558 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TomeUnmerge] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.711 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamWV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.861 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DiagFlat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.435.991 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.143 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedBiasAddGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.305 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeComputeExpertTokens] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.466 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRouting] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.584 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2Grad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.692 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NanToNum] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.436.807 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.133 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAbs] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.163 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCopy] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.186 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.208 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAcos] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.230 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.256 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.280 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.305 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.325 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.346 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcdivScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.372 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.395 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.416 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddcmulScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.436 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAsin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.457 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAtan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.478 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCos] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.499 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachCosh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.519 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.541 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.562 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.582 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.603 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachErfc] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.624 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExp] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.644 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachExpm1] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.664 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.700 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLerpScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.723 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.744 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog1p] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.766 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.785 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLog10] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.802 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.823 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.843 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMaximumScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.864 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.887 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.909 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMinimumScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.931 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.955 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.976 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.437.997 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachNeg] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.018 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.039 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.060 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarAndTensor] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.081 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachPowScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.101 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachReciprocal] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.122 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachRoundOffNumber] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.143 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSigmoid] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.163 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSin] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.184 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSinh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.206 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSqrt] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.236 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.258 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalar] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.281 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarList] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.303 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTan] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.324 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachTanh] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.344 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachZeroInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachACosInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.647 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachAddListInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.819 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulScalarInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.846 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachLogInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.869 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubListInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.891 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachMulListInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.438.913 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachDivListInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.067 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachSubScalarInplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdvanceStep] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.286 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Logit] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.394 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LogitGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FatreluMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.616 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GeluMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.777 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.439.894 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyAdamV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.052 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ApplyFusedEmaAdam] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.188 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AxpyWithSoftmaxAndDropOutDoMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.302 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoU] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.330 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoU] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.367 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoU] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.470 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DIoUGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.497 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CIoUGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.521 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GIoUGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.634 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Cross] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.792 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.440.944 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CrossEntropyLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.085 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FeedsRepeat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.198 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.388 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IndexPutWithSort] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.497 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.685 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceIndexAddWithSorted] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.799 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Iou] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.910 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.936 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormReduceV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.441.958 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LpNormV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.112 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantRopeQuantKvcache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.264 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantBias] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.418 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermuteGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.566 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermuteGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenPermute] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTokenUnpermute] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.442.997 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MulAddn] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.153 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwish] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.312 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupNormSwishGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.489 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttentionGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.625 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiScaleDeformableAttnFunction] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.736 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingGradV2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.844 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PSROIPoolingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.443.955 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Renorm] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.063 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleave] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.235 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RepeatInterleaveGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.385 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RingAttentionUpdate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.501 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Resize] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.616 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.723 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeGradD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.836 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.444.943 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryMulGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.445.369 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbedding] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.445.804 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPositionEmbeddingGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.445.925 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RopeWithSinCosCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.044 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedIou] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.071 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedOverlaps] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.177 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.289 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.396 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsPack] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.510 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SignBitsUnpack] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.626 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SyncResizeBilinearV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.738 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ViewCopy] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.843 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMultiClassNonMaxSuppression] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.446.947 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.118 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AmpUpdateScale] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.231 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BallQuery] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.343 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistance] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.525 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ChamferDistanceGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPool] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.746 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeformableRoiPoolGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.447.863 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PointsInPolygons] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.046 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotated] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.271 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiAlignRotatedGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.388 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingGradWithArgMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.505 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiPoolingWithArgMax] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.612 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlign] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.714 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotatedFeatureAlignGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.823 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLoss] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.448.934 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SigmoidFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.047 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SoftmaxFocalLossGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StackBallQuery] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.322 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolate] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.472 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ThreeInterpolateBackward] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.730 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScoreGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.449.919 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionFFN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.085 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScoreQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.265 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQkvQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.683 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwinTransformerLnQKV] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.891 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.916 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.938 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.958 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.450.983 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.451.005 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV2CompressDequant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.451.027 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatmulFixpipe] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.451.048 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchmatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.451.781 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatMulV3] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.452.190 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulV3] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.452.604 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransposeBatchMatMul] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.452.798 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.452.992 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantBatchMatmulV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.453.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TransQuantParamV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.453.519 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.453.682 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.456.214 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[WeightQuantBatchMatmulV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.458.223 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynSeqOuter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.458.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Expand] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.458.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MovingSumWithSigmoid] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.458.582 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[YoloxBoundingBoxDecode] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.458.872 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CircularPad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.021 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[CircularPadGrad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.213 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RotaryPosEmbInfer] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.548 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.592 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNz] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.617 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheNdSiso] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.642 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompress] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.672 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReshapeAndCacheCompressRope] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.847 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleLinear1d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.459.874 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.035 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.215 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.383 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dAAGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.543 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBicubic2dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.717 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.744 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.460.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAA] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.072 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dAABackward] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.232 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleBilinear2dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.396 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ResizeUpsampleTrilinear] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.557 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleTrilinear3dBackward] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.707 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest3d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.733 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact3d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.461.882 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.056 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearestExact2dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.086 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[UpsampleNearest2dGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.193 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.315 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.351 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.473 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2DBackpropInput] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.605 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTranspose] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.718 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.835 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInput] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.462.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTranspose] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.463.064 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Deconvolution] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.463.167 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Dilation] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.463.356 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.463.552 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropFilterV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.463.911 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropFilterV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.464.290 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.464.487 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DBackpropInputV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.464.808 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DTransposeV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.464.993 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2DTransposeV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.201 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ReflectionPad3dGrad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.325 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.441 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPoolV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.584 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.610 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DepthwiseConv2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Pooling] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.465.660 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantConv2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.466.748 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveMaxPool3d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.188 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AdaptiveAvgPool3d] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.357 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.514 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.659 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.688 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.838 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormDynamicQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.467.987 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.138 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.180 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.336 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[KvRmsNormRopeCache] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.489 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SingleRope] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.642 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddRmsNormCast] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.805 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AddLora] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.468.953 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AGLU] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.469.332 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AllGatherMatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.469.689 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AlltoAllAllGatherBatchMatMul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.469.916 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[AvgPool3DGrad] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.470.100 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BackgroundReplace] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.471.153 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BlendImagesCustom] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.471.570 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Conv3DV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.099 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NLLLossGrad310p] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.286 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.439 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DeepNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.551 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DropoutWithMulsAndSoftmaxGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.709 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DuaQuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.859 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.472.897 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DynamicQuantV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.015 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ExtractImagePatches] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.042 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Im2col] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.369 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsFinite] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.480 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IsInf] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.637 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ForeachBinaryOp] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.796 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlatQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.473.909 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Ger] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.207 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatMulAllReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.390 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulAdd] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.547 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.662 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IFMR] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.770 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InTopKD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.474.924 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InstanceNormV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.101 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IouV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.278 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BidirectionLSTM] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.306 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BidirectionLSTMV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.464 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantMatmulDequant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.491 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantGroupedMatmulDequant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSelectV3] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.475.821 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MaskedSoftmaxWithRelPosBias] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.477.348 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.483.095 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.483.141 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[InplaceMatmulAllReduceAddRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.485.945 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulLayerNormReduce] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.486.202 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MatmulReduceScatter] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.486.498 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MedianDim] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.486.820 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[BatchMatMulReduceScatterAlltoAll] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.027 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeDispatch] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.232 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeDistributeCombine] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.393 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeFinalizeRoutingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.544 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelCombineGates] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.731 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelCombineX] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.875 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeTutelDispatch] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.487.967 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MrgbaCustom] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.488.233 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MultiHeadAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.488.353 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[NMSWithMask] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.488.513 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantizeAddLayerNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.488.994 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Rfft1D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.141 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.297 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.326 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GemmaRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.440 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RoiExtractor] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.558 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterElements] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.737 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinearIndex] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.489.931 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[LinearIndexV2] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.113 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScatterAddWithSorted] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.262 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[Sinkhorn] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.377 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SortedNMS] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.484 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrlD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.510 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SparseApplyFtrl] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.731 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SquareSumAll] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.490.922 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StackGroupPoints] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.067 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[StridedSliceAssignV2] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.173 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SubMulConcat] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.326 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGlu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.358 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluGrad] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.479 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[SwiGluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.595 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKD] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.619 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2D] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.640 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.780 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[TopKV3] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.491.979 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.069 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MseLossGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.310 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[ScaledMaskedSoftmaxGradV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.504 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNorm] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.530 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PreRmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.554 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[RmsNormQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.744 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[QuantPerChannel] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.492.773 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DequantPerChannel] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.440 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DBackpropFilter, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.522 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DBackpropInput, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.546 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2DTranspose, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.618 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv2D, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.692 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3DBackpropInput, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.714 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3DTranspose, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.783 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: Conv3D, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.962 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: FFN, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.495.983 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatmulTunnerTiling, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.097 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulV2, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.119 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: BatchMatmulFixpipe, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.137 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulV3, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.167 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: QuantBatchMatmulV3, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.329 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: DynamicRNN, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.496.350 [tuning_tiling_registry.cc:30]619427 RegisterTilingData:optype: MatMulTunnerTiling, registered count: 15 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.497.473 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:op_select_format, op_type:MatMulV2, funcPointer:0xfffee7b3d0b0, registered count:7 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.497.503 [op_check.cc:24]619427 RegisterOpCapability:RegisterOpCapability: check_type:check_supported, op_type:MatMulV2, funcPointer:0xfffee7b3d310, registered count:5 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.497.901 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.498.400 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectFlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.500.026 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[DetectMatMul] register OP_IMPL : [][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.500.205 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.500.291 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FFN] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.500.660 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.501.098 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FlashAttentionScoreGrad] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.531.988 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.532.092 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[FusedInferAttentionScore] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.532.274 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.532.359 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmul] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.532.505 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwiglu] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.532.638 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[GroupedMatmulSwigluQuant] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.533.315 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.533.408 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[IncreFlashAttention] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.533.665 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaProlog] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.533.744 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MlaAttentionScore] register OP_IMPL : [] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.533.893 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[MoeInitRoutingV2] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.534.125 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.534.297 [op_impl_registry.cc:122]619427 RegisterOpImplToRegistry:Op type[PromptFlashAttention] register OP_IMPL : [][][][] [INFO] GE(619427,python3.9):2025-04-03-09:35:44.534.957 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.540.905 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpNum: 599 [INFO] GE(619427,python3.9):2025-04-03-09:35:44.542.945 [op_impl_registry_holder_manager.cc:208]619427 GetOpImplFunctionsByHandle:GetRegisteredOpCtNum: 0 [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.619.035 [op_impl_space_registry.cc:109]619427 MergeFunctions:op type AvgPool3DGrad inputs_dependency has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.620.391 [op_impl_space_registry.cc:84]619427 MergeFunctions:op type MoeInitRoutingV2 tiling_parse func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.620.427 [op_impl_space_registry.cc:85]619427 MergeFunctions:op type MoeInitRoutingV2 compile_info_creator func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.620.439 [op_impl_space_registry.cc:86]619427 MergeFunctions:op type MoeInitRoutingV2 compile_info_deleter func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.620.454 [op_impl_space_registry.cc:87]619427 MergeFunctions:op type MoeInitRoutingV2 tiling func has been registered [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.620.467 [op_impl_space_registry.cc:93]619427 MergeFunctions:op type MoeInitRoutingV2 max_tiling_data_size has been registered [INFO] GE(619427,python3.9):2025-04-03-09:35:44.621.497 [op_impl_space_registry.cc:273]619427 ConvertSoToRegistry:Save so symbol and handle in path[/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe/op_tiling/lib/linux/aarch64//libopmaster_rt.so] success! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.019 [plugin_manager.cc:564]619427 GetUpgradedOpMasterPath:Start to get upgraded op tiling path [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.048 [plugin_manager.cc:134]619427 GetUpgradedOppPath:Enter get upgraded opp path schedule [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.622.113 [file_utils.cc:47]619427 RealPath:[Util][realpath] Can not get real_path for /usr/local/Ascend/ascend-toolkit/latest/opp_latest/, reason:No such file or directory [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.622.132 [plugin_manager.cc:145]619427 GetUpgradedOppPath:[Call][RealPath] Opp path /usr/local/Ascend/ascend-toolkit/latest/opp_latest/ is invalid. [WARNING] GE(619427,python3.9):2025-04-03-09:35:44.622.146 [plugin_manager.cc:567]619427 GetUpgradedOpMasterPath:Failed to get upgraded opp path! [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.224 [mem_manager.cc:31]619427 Initialize:MemManager Initialize. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.261 [rts_caching_mem_allocator.cc:131]619427 Initialize:RtsAllocatorManager Initialize. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.279 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.295 [graph_mem_allocator.cc:17]619427 Initialize:MemoryAllocator::Initialize. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.335 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.352 [graph_mem_allocator.cc:17]619427 Initialize:MemoryAllocator::Initialize. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.366 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.381 [caching_allocator.cc:391]619427 FreeBlocks:Free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.394 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.412 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.426 [caching_allocator.cc:391]619427 FreeBlocks:Free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.436 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.451 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.467 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.479 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.494 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.507 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[2] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.522 [session_scope_mem_allocator.cc:73]619427 FreeAllMemory:Free all memory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.537 [mem_manager.cc:102]619427 InitAllocator:Create Allocator memory type[17] success. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.549 [session_scope_mem_allocator.cc:73]619427 FreeAllMemory:Free all memory [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.601 [dump_manager.cc:152]619427 Init:exception dump is not enabled [INFO] GE(619427,python3.9):2025-04-03-09:35:44.622.659 [ge_executor.cc:315]619427 Initialize:Init GeExecutor over. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.680 [acl.cpp:321]619427 aclInit: set DumpConfig in aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.697 [dump.cpp:841]619427 HandleDumpConfig: start to execute HandleDumpConfig. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.866 [dump.cpp:759]619427 ConvertDumpCfg: start to execute ConvertDumpCfg. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.883 [dump.cpp:594]619427 IsValidDumpConfig: start to execute IsValidDumpConfig. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.895 [dump.cpp:224]619427 CheckDumpSceneSwitch: start to execute CheckDumpSceneSwitch. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.929 [dump.cpp:703]619427 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.948 [dump.cpp:771]619427 HandleDumpCommand: start to execute HandleDumpCommand. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.622.963 [dump.cpp:703]619427 SetUpDumpConfig: convert to ge dump config successfully, enable lite_exception dump, path= [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.046 [adx_datadump_server.cpp:77][tid:619427] start to do dump init [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.067 [adx_dump_record.cpp:210][tid:619427] dumpPath prefix is /home/jenkins/mindspore/testcases/testcases/tests/st/lccl [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.085 [adx_dump_record.cpp:214][tid:619427] record remote dump temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/ [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.335 [adx_server_manager.cpp:149][tid:619632] server manager add component (10) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.363 [adx_dump_record.cpp:627][tid:619633] start dump thread, remote dump record temp path : /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.371 [adx_server_manager.cpp:165][tid:619632] server manager components init successfully [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.522 [adx_server_manager.cpp:203][tid:619634] Run Server(0) Process [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.576 [adx_server_manager.cpp:369][tid:619634] device up 0 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.594 [hdc_comm_opt.cpp:59][tid:619634] open device[0] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.623.708 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=0; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.623.727 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=0) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.745 [hdc_api.cpp:119][tid:619634] logDevId 0 create HDC server successfully [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.790 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.806 [adx_server_manager.cpp:369][tid:619634] device up 1 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.817 [hdc_comm_opt.cpp:59][tid:619634] open device[1] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.623.946 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=1; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.623.963 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=1) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.976 [hdc_api.cpp:119][tid:619634] logDevId 1 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.623.990 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.004 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.017 [adx_server_manager.cpp:369][tid:619634] device up 2 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.028 [hdc_comm_opt.cpp:59][tid:619634] open device[2] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.098 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=2; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.116 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=2) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.133 [hdc_api.cpp:119][tid:619634] logDevId 2 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.146 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.177 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.195 [adx_server_manager.cpp:369][tid:619634] device up 3 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.206 [hdc_comm_opt.cpp:59][tid:619634] open device[3] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.333 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=3; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.351 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=3) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.378 [hdc_api.cpp:119][tid:619634] logDevId 3 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.389 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.403 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.417 [adx_server_manager.cpp:369][tid:619634] device up 4 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.427 [hdc_comm_opt.cpp:59][tid:619634] open device[4] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.556 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=4; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.572 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=4) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.585 [hdc_api.cpp:119][tid:619634] logDevId 4 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.597 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.614 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.629 [adx_server_manager.cpp:369][tid:619634] device up 5 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.638 [hdc_comm_opt.cpp:59][tid:619634] open device[5] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.755 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=5; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.771 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=5) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.784 [hdc_api.cpp:119][tid:619634] logDevId 5 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.796 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.811 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.821 [adx_server_manager.cpp:369][tid:619634] device up 6 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.833 [hdc_comm_opt.cpp:59][tid:619634] open device[6] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.963 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=6; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.624.979 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=6) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.624.991 [hdc_api.cpp:119][tid:619634] logDevId 6 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.003 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.018 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.034 [adx_server_manager.cpp:369][tid:619634] device up 7 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.046 [hdc_comm_opt.cpp:59][tid:619634] open device[7] server[13] [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.625.168 [drv_log_user.c:623][ascend][curpid:619427,619634][drv][hdc][share_log_read_in_single_module]Server create success.(dev_id=7; fid=0; service_type="service_dump"; pid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.625.185 [hdc_server.c:363][ascend][curpid:619427,619634][drv][hdc][drvHdcServerCreate]Create server success. (listen_device=7) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.207 [hdc_api.cpp:119][tid:619634] logDevId 7 create HDC server successfully [WARNING] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.217 [adx_hdc_epoll.cpp:34][tid:619634] hdc epoll has been created. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.625.231 [adx_server_manager.cpp:111][tid:619634] create server info [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.626.459 [adx_server_manager.cpp:457][tid:619427] The server is initialized after waiting 3 times. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.626.483 [ge_executor.cc:1349]619427 SetDump:Start to set dump config [INFO] GE(619427,python3.9):2025-04-03-09:35:44.626.499 [dump_manager.cc:201]619427 NeedDoDump:Dump status: on, Overflow detection dump switch: off. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.626.541 [dump_manager.cc:261]619427 SetExceptionDump:Enable brief exception dump. [INFO] GE(619427,python3.9):2025-04-03-09:35:44.626.577 [ge_executor.cc:1355]619427 SetDump:Set dump config successfully [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.626.617 [dump_manager.cpp:146][tid:619427] Set 3 dump setting, status: on, dump switch: 3 [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:44.627.641 [dump_manager.cpp:301][tid:619427] Register exception callback, registered: 1 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.663 [dump.cpp:825]619427 HandleDumpCommand: set dump config for model successfully,adumpType = 3, dump_path = , dump_mode = , dump_op_switch = , dump_data = . [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.688 [acl.cpp:327]619427 aclInit: set HandleDumpConfig success in aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.703 [acl.cpp:329]619427 aclInit: set max_opqueue_num in aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.719 [acl_resource_manager.cpp:248]619427 HandleMaxOpQueueConfig: start to execute HandleMaxOpQueueConfig [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.831 [acl_resource_manager.cpp:275]619427 HandleMaxOpQueueConfig: no max_opqueue_num found, set default DEFAULT_MAX_OPQUEUE_NUM[20000]. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.851 [acl_resource_manager.cpp:278]619427 HandleMaxOpQueueConfig: HandleMaxOpQueueConfig end in HandleMaxOpQueueConfig [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.863 [acl.cpp:335]619427 aclInit: set HandleMaxOpQueueConfig success in aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.875 [acl.cpp:336]619427 aclInit: set HandleDefaultDevice in aclInit [WARNING] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.942 [json_parser.cpp:260]619427 GetDefaultDeviceIdFromFile: no defaultDevice item! [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.627.960 [acl.cpp:343]619427 aclInit: set HandleDefaultDevice success in aclInit [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:44.627.987 [prof_atls_plugin.cpp:210] (tid:619427) Module[48] register callback of ctrl handle. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.005 [acl.cpp:351]619427 aclInit: set ProfilingConfig in aclInit [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.017 [profiling.cpp:141]619427 HandleProfilingConfig: start to execute HandleProfilingConfig [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.033 [profiling.cpp:135]619427 GetProfilingConfigFile: no profiling config file. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.096 [profiling.cpp:167]619427 HandleProfilingConfig: can not parse profiling config from file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619427.json], errorCode = 0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.113 [profiling.cpp:172]619427 HandleProfilingConfig: profiling config file[/home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619427.json] is empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.126 [profiling.cpp:176]619427 HandleProfilingConfig: start to use profiling config by config file mode [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.140 [profiling.cpp:179]619427 HandleProfilingConfig: ParseJsonFromFile ok in HandleProfilingConfig [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.196 [profiling.cpp:92]619427 HandleProfilingCommand: start to execute HandleProfilingCommand [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:44.628.611 [ai_drv_dev_api.cpp:316] >>> (tid:619427) Succeeded to get device 64 frequency=100000 [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:44.628.729 [msprof_callback_impl.cpp:132] >>> (tid:619427) Dynamic profiling environment not really set, so do nothing. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.747 [profiling.cpp:122]619427 HandleProfilingCommand: set profiling config successfully [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.759 [profiling.cpp:186]619427 HandleProfilingConfig: set HandleProfilingConfig success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.772 [hash_utils.cpp:108]619427 CalculateSimpleHash: Begin to calculate hash for file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619427.json [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.815 [hash_utils.cpp:128]619427 CalculateSimpleHash: file: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl/aclinit_619427.json, hash: 3917491750093904922 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.834 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.871 [acl.cpp:280]619427 IsEnableAutoUCMemeory: auto-uc-memory is enabled. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.887 [acl.cpp:385]619427 aclInit: register kernel launch fill function in aclInit [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.628.908 [runtime.cc:5394] 619427 RegKernelLaunchFillFunc: registercallback finish, symbol:g_opSystemRunCfg. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.628.926 [api_impl.cc:7407] 619427 RegStreamStateCallback: Reg stream state callback, regName=ACL_MODULE_STREAM. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.628.956 [api_impl.cc:4211] 619427 RegDeviceStateCallback: Reg device state callback, regName=ACL_MODULE_DEVICE. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.628.975 [acl.cpp:420]619427 aclInit: successfully execute aclInit, aclInitRefCount is 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.629.170 [mindspore/ccsrc/plugin/res_manager/ascend/hal_manager/ascend_hal_manager.cc:199] InitializeAcl] Call aclInit successfully, json is {"dump":{"dump_scene":"lite_exception"},"err_msg_mode":"1"} [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.629.316 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:295] Initialize] End initializing device context. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.629.387 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:467] LoadCollectiveCommLib] Loading MACCL collective library successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.629.413 [mindspore/ccsrc/distributed/collective/collective_manager.cc:621] InitDeviceCommLib] Start initializing communication library on device side... [2025-04-03 09:35:44.634726] [info] [619427] [profiling_funcs.cpp:47] MsprofRegisterCallback start! mki_log log dir:/home/jenkins/atb/log exist [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:44.635.423 [prof_atls_plugin.cpp:210] (tid:619427) Module[0] register callback of ctrl handle. [WARNING] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.636.295 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:76] Initialize] Loading LCCL because env MS_ENABLE_LCCL is set to on. Pay attention that LCCL only supports communication group within single node in KernelByKernel for now. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.636.330 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:80] Initialize] Successfully initialize LCCL. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.636.433 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:83] OpenTsd] Device id = 2, rank size = 8. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.636.455 [device.cpp:143]619427 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.485 [api_impl.cc:2999] 619427 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.504 [device.cc:306] 619427 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.523 [runtime.cc:3393] 619427 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.545 [runtime.cc:3560] 619427 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.565 [api_impl.cc:2987] 619427 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.582 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.603 [api_impl.cc:2992] 619427 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.627 [api_impl.cc:3025] 619427 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.636.643 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.636.659 [acl.cpp:121]619427 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.636.687 [device.cpp:154]619427 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.636.745 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel _npu_log begins the construction process witch capacity 128 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.636.766 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is _npu_log, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.790 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.807 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.636.824 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.636.856 [queue_client.c:151][ascend][curpid:619427,619427][drv][queuemng][QueueGetHostpid]Query tgid. (tgid=619427) [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.637.043 [queue_client.c:305][ascend][curpid:619427,619427][drv][queuemng][QueueGetDevPid]Query devpid succ. (devId=2; vfid=0; hostpid=619427; master_pid=619427; proc_type=0; devpid=6117). [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.641.150 [queue_client.c:362][ascend][curpid:619427,619427][drv][queuemng][QueueInit]QueueInit device. (devid=2; mcast_flag=0; mcast_gid=0; proc_sn=0; event_sn_offset=0) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.365 [npu_driver.cc:4114] 619427 MemQueueInit: get max queue cnt 4294967294 success. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.389 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.405 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=_npu_log. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.641.633 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=0, actualQid=0, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.641.652 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is _npu_log, real name is _npu_log, qid is 0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.641.847 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.641.886 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.906 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.924 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.641.927 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_dump begins the construction process witch capacity 128 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.641.955 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_tensor_dump, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.980 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.641.995 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.006 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.073 [npu_driver.cc:4087] 619427 MemQueueInit: repeated init [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.091 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.107 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=ms_tensor_dump. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.642.314 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=1, actualQid=1, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.642.333 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_tensor_dump, real name is ms_tensor_dump, qid is 1 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.642.466 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.642.502 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.642.513 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_tensor_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.522 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.642.536 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_tensor_summary, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.539 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.558 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.571 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.586 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.626 [npu_driver.cc:4087] 619427 MemQueueInit: repeated init [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.653 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.642.666 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=ms_tensor_summary. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.642.826 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=2, actualQid=2, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.642.845 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_tensor_summary, real name is ms_tensor_summary, qid is 2 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.047 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.075 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.643.093 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_image_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.097 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.120 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.117 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_image_summary, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.145 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.160 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.172 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.210 [npu_driver.cc:4087] 619427 MemQueueInit: repeated init [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.226 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.242 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=ms_image_summary. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.643.388 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=3, actualQid=3, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.405 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_image_summary, real name is ms_image_summary, qid is 3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.532 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.568 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.643.586 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_scalar_summary begins the construction process witch capacity 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.590 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.608 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_scalar_summary, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.611 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.641 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.654 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.667 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.707 [npu_driver.cc:4087] 619427 MemQueueInit: repeated init [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.723 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.643.739 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=ms_scalar_summary. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.643.894 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=4, actualQid=4, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.643.913 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_scalar_summary, real name is ms_scalar_summary, qid is 4 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.042 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.644.069 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:308] MbufDataHandler] Channel ms_histogram_summary begins the construction process witch capacity 128 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.080 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.089 [tensor_data_transfer.cpp:898]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_histogram_summary, capacity is 128 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.102 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.108 [api_impl.cc:6077] 619427 MemQueueInit: Start to init queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.117 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.123 [npu_driver.cc:4076] 619427 MemQueueInit: init mem queue, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.137 [npu_driver.cc:4026] 619427 CheckMemQueueSupported: get majorVersion 7 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.184 [npu_driver.cc:4087] 619427 MemQueueInit: repeated init [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.204 [api_impl.cc:6040] 619427 MemQueueCreate: Start to create queue on devId 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.216 [npu_driver.cc:3931] 619427 MemQueueCreate: create mem queue, deviceId=2, name=ms_histogram_summary. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.644.359 [queue_interface.c:182][ascend][curpid:619427,619427][drv][queuemng][halQueueCreate]Create queue successful. (virtualQid=5, actualQid=5, deploy_type=0) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.377 [tensor_data_transfer.cpp:940]619427 acltdtCreateChannelWithCapacity: acltdtCreateChannelWithCapacity devId is 2, name is ms_histogram_summary, real name is ms_histogram_summary, qid is 5 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.503 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.644.543 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.644.560 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:273] InitHccl] Start init hccl adapter. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.565 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.644.585 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.644.764 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:569] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.644.798 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:586] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.818 [plugin_manager.cc:25] [619427]hcom ops plugin init start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.855 [plugin_manager.cc:38] [619427]input BUILD_MODE is not set, result autoTune is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.877 [plugin_manager.cc:42] [619427]hcom running normal mode. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.926 [hcom_plugin.cc:50] [619427]Initialize start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.943 [hcom_plugin.cc:52] [619427]initializeCount_ : 1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.644.980 [hcom_plugin.cc:60] [619427]hvdOpsKernelInfoStoreInfoPtr_ is initialized [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.007 [hcom_plugin.cc:63] [619427]hvdGraphOptimizerPtr_ is initialized [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.023 [hcom_plugin.cc:329] [619427]ParserHcclAlgoDesc: there is no key named "HCCL_algorithm" in the options. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.041 [hcom_plugin.cc:281] [619427]ParserHcclExecTimeOut: there is no key named "HCCL_execTimeOut" in the options. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:44.645.064 [hcom_plugin.cc:217] [619427]option profiling mode is not found. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.082 [hcom_plugin.cc:164] [619427][masterInfo]get Master Info:ip or port or deviceId not set. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.098 [hcom_plugin.cc:180] [619427][masterInfo]get Rank Info: worker size not set. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.110 [hcom_plugin.cc:244] [619427]InitializePlugin: Init without ranktable and masterInfo, please check your mode. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.238 [hcom_plugin.cc:82] [619427]hccl ops plugin init success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.256 [plugin_manager.cc:48] [619427]hcom ops plugin init end. ret[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.270 [hcom_plugin.cc:126] [619427]get hccl kernel info store start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.296 [hcom_plugin.cc:132] [619427]get hccl kernel info store finished. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.645.326 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:608] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.645.351 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:786] InitHcclExec] Start init hccl exec. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.365 [hcom_executor.cc:29] [619427]Hcom Excutor Initialize start. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.420 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.440 [adapter_rts.cc:234] [619427][hrtGetDevice]deviceLogicId[2] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.479 [api_c_soc.cc:70] 619427 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.530 [hcom_common.cc:85] [619427][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.561 [hcom_common.cc:88] [619427][HcomGetCurHcomCtx] is no Used deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.588 [hcom.cc:486] [619427]Get Hcom Init Status: [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.605 [hcom_common.cc:85] [619427][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.645.621 [hcom_rpc.cc:1019] [619427]Get Hcom Init Status: [0] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:44.645.646 [hcom_base_funs.cc:36] [619427][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_RANK_TABLE is not found. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:44.645.666 [hcom_executor.cc:121] [619427][HcomOpsKernelInfoStore][InitHcom]OPTION_EXEC_HCOM_GROUPLIST is not found. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.736 [api_impl.cc:1120] 619427 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.766 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.857 [stream.cc:2246] 619427 AllocExecutedTimesSvm: stream_id=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.930 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.645.948 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.257 [npu_driver.cc:6125] 619427 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.308 [npu_driver.cc:6137] 619427 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1008, streamId=4, logicCq=1008, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.337 [npu_driver.cc:6189] 619427 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=4, logicCq=1008. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.357 [npu_driver.cc:5916] 619427 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0186000, addr=dfffe0186000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.374 [stream.cc:653] 619427 Setup: Success to get sq=4 sq reg virtual addr length=4096. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.646.451 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.471 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.646.830 [stream.cc:291] 619427 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=4, flags_=0x600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.426 [stream.cc:688] 619427 Setup: stream setup end, stream_id=4, IsTaskSink=0, sqId=4, cqId=4, deviceId=2, streamResId=65535 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.647.448 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.647.464 [acl_resource_manager.cpp:816]619427 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.481 [api_impl.cc:1162] 619427 StreamCreate: Succ, flags=1536, stream_id=4, context=aaaae7011ab0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.647.512 [stream.cc:73] [619427]Construct stream by stream type success, ptr[0xaaaaf241f0c0] ctx[0xaaaae7011ab0], stream id[4], cqId[4], logicCqid[1008] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.553 [api_impl.cc:1120] 619427 StreamCreate: priority=0, flags=1536. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.592 [npu_driver.cc:404] 619427 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.623 [stream.cc:2246] 619427 AllocExecutedTimesSvm: stream_id=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.659 [stream_sqcq_manage.cc:147] 619427 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.676 [npu_driver.cc:882] 619427 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.841 [npu_driver.cc:6125] 619427 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.889 [npu_driver.cc:6137] 619427 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1009, streamId=5, logicCq=1009, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.911 [npu_driver.cc:6189] 619427 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=5, logicCq=1009. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.929 [npu_driver.cc:5916] 619427 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0165000, addr=dfffe0165000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.647.944 [stream.cc:653] 619427 Setup: Success to get sq=5 sq reg virtual addr length=4096. [INFO] DRV(619427,python3.9):2025-04-03-09:35:44.647.989 [devmm_svm.c:1972][ascend][curpid:619427,619427][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0xa; out_feature=0xa) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.648.008 [npu_driver.cc:6080] 619427 CheckSupportPcieBarCopy: chip type=5, drv support pcie bar copy [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.648.335 [stream.cc:291] 619427 CreateStreamTaskRes: STREAM TaskRes Setup stream_id=5, flags_=0x600. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.648.979 [stream.cc:688] 619427 Setup: stream setup end, stream_id=5, IsTaskSink=0, sqId=5, cqId=5, deviceId=2, streamResId=65535 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.649.002 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.649.018 [acl_resource_manager.cpp:816]619427 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.031 [api_impl.cc:1162] 619427 StreamCreate: Succ, flags=1536, stream_id=5, context=aaaae7011ab0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.649.052 [stream.cc:73] [619427]Construct stream by stream type success, ptr[0xaaaaf249b120] ctx[0xaaaae7011ab0], stream id[5], cqId[5], logicCqid[1009] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:44.649.311 [sal.cc:619] [619647]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.649.344 [hcom_executor.cc:587] [619647]MessageProcessThreadLoop start queType:[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.371 [api_impl.cc:2999] 619647 SetDevice: device_id=2, deviceMode=0. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:44.649.374 [sal.cc:619] [619648]err[34] link[Hccl_MessageProcess] nameSet failed. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.387 [device.cc:306] 619647 Init: device mode has been already set, mode=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.649.402 [hcom_executor.cc:587] [619648]MessageProcessThreadLoop start queType:[1] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.408 [runtime.cc:3393] 619647 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.424 [api_impl.cc:2999] 619648 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.424 [runtime.cc:3560] 619647 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.429 [api_impl.cc:2999] 619649 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.441 [device.cc:306] 619648 Init: device mode has been already set, mode=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:44.649.443 [hcom_executor.cc:34] [619427][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.446 [api_impl.cc:2987] 619647 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.487 [runtime.cc:3393] 619648 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.489 [device.cc:306] 619649 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.487 [api_impl.cc:2999] 619650 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.502 [runtime.cc:5239] 619647 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.526 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:792] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.508 [runtime.cc:3560] 619648 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.515 [runtime.cc:3393] 619649 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.551 [api_impl.cc:2987] 619648 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.532 [device.cc:306] 619650 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.545 [api_impl.cc:2992] 619647 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.553 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:798] InitHcclExec] InitHcclExec success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.560 [runtime.cc:3560] 619649 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.564 [runtime.cc:5239] 619648 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.571 [runtime.cc:3393] 619650 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x6 [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.588 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:291] InitHccl] Init hccl adapter success. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:44.649.586 [awatchdog_core.c:175](tid:619647) create watchdog for id : 65543, tid : 619647, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.591 [api_impl.cc:2987] 619649 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.601 [api_impl.cc:2992] 619648 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.604 [runtime.cc:3560] 619650 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.618 [api_impl.cc:3025] 619647 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.622 [runtime.cc:5239] 619649 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.626 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:92] Initialize] Successfully initialize HCCL. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:44.649.634 [awatchdog_core.c:175](tid:619648) create watchdog for id : 65543, tid : 619648, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.635 [api_impl.cc:2987] 619650 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.653 [api_impl.cc:2992] 619649 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.667 [mindspore/ccsrc/distributed/collective/collective_manager.cc:628] InitDeviceCommLib] Communication library on device side is successfully initialized. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.673 [api_impl.cc:3025] 619648 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.677 [runtime.cc:5239] 619650 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:44.649.690 [awatchdog_core.c:175](tid:619649) create watchdog for id : 65543, tid : 619649, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.708 [api_impl.cc:2992] 619650 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.709 [api_impl.cc:3025] 619649 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.718 [mindspore/ccsrc/distributed/collective/collective_manager.cc:211] Initialize] [PROF]InitDeviceBackend costs 1141.15 msec. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:44.649.729 [awatchdog_core.c:175](tid:619650) create watchdog for id : 65543, tid : 619650, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.649.744 [api_impl.cc:3025] 619650 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.755 [mindspore/ccsrc/distributed/collective/collective_manager.cc:860] IsAsyncInitGlobalComm] Async initialize global comm: 0. async_conf: 1, is_graph: 0, use_rank_table: 0, simulation: 0, use_mpi: 0, is_ascend: 1 [WARNING] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.777 [mindspore/ccsrc/distributed/collective/collective_manager.cc:335] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3, 4, 5, 6, 7}, async: 0, submit_now: 1 [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.838 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnHostSide costs 0.017 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.918 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:148] CreateCommunicationGroup] Successfully create LCCL communication group hccl_world_group [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.959 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_collective_comm_lib.cc:166] CreateCommunicationGroup] Successfully create HCCL communication group hccl_world_group [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.649.984 [mindspore/ccsrc/distributed/collective/collective_manager.cc:374] CreateCommunicationGroup] [PROF]CreateCommunicationGroupOnDeviceSide costs 0.122 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.650.114 [mindspore/ccsrc/distributed/collective/collective_manager.cc:919] SubmitCreateDeviceCommTask] Launch init comm thread. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:44.650.141 [mindspore/ccsrc/distributed/collective/collective_manager.cc:925] SubmitCreateDeviceCommTask] Submit init communicator task for hccl_world_group. Call 'WaitCommInitDone' later to wait initialization to be done. [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:44.650.213 [mindspore/ccsrc/distributed/collective/collective_manager.cc:973] RunInitCommTasks] Create device communicator in thread for group: hccl_world_group [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:44.650.257 [mindspore/ccsrc/distributed/collective/collective_manager.cc:764] CreateDeviceCommunicator] Create device communicator for hccl_world_group [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.650.304 [device.cpp:143]619651 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.336 [api_impl.cc:2999] 619651 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.353 [device.cc:306] 619651 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.370 [runtime.cc:3393] 619651 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.387 [runtime.cc:3560] 619651 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.403 [api_impl.cc:2987] 619651 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.417 [runtime.cc:5239] 619651 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.432 [api_impl.cc:2992] 619651 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:44.650.453 [awatchdog_core.c:175](tid:619651) create watchdog for id : 65543, tid : 619651, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.650.468 [api_impl.cc:3025] 619651 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.650.485 [acl.cpp:115]619651 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.650.501 [acl.cpp:121]619651 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.650.516 [device.cpp:154]619651 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:44.650.549 [mindspore/ccsrc/distributed/collective/collective_manager.cc:787] CreateDeviceCommunicator] [PROF]GenerateRootInfo costs 0.242 msec. [WARNING] DEVICE(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:44.650.915 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:248] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 399/400, sleep 2 [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.804.178 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.804.178 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.804.215 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:44.804.223 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.230 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.243 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.252 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.258 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.804.271 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.804.271 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.804.303 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:44.804.303 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.324 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.804.324 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.972.745 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:44.972.779 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:44.972.797 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:44.972.810 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.972.830 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:44.972.910 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:45.151.144 [mindspore/ccsrc/distributed/collective/collective_manager.cc:806] CreateDeviceCommunicator] Successfully send/fetch unqiueid for communication group hccl_world_group [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:45.151.187 [mindspore/ccsrc/distributed/collective/collective_manager.cc:808] CreateDeviceCommunicator] [PROF]BroadcastUniqueID costs 500.606 msec. [WARNING] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:45.151.205 [mindspore/ccsrc/distributed/collective/collective_manager.cc:817] CreateDeviceCommunicator] Begin initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:45.151.228 [mindspore/ccsrc/distributed/collective/collective_manager.cc:744] GetCommunicatorInitTimeout] HCCL_CONNECT_TIMEOUT is 600 seconds. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.151.406 [device.cpp:143]619758 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.438 [api_impl.cc:2999] 619758 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.453 [device.cc:306] 619758 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.466 [runtime.cc:3393] 619758 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x8 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.479 [runtime.cc:3560] 619758 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.491 [api_impl.cc:2987] 619758 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.503 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.519 [api_impl.cc:2992] 619758 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:45.151.535 [awatchdog_core.c:175](tid:619758) create watchdog for id : 65543, tid : 619758, timeout : 300s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.151.557 [api_impl.cc:3025] 619758 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.151.570 [acl.cpp:115]619758 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.151.580 [acl.cpp:121]619758 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.151.591 [device.cpp:154]619758 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.308.313 [device.cpp:231]619758 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.362 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.522 [api_c_soc.cc:70] 619758 rtGetSocVersion: soc version is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.308.593 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.614 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.637 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.650 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.665 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.680 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=0. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.308.702 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=0.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.308.737 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.750 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.762 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.774 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.788 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.308.802 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=1. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.308.977 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=1.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.309.016 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.036 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.053 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.065 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.079 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.092 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=3. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.309.554 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=3.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.309.590 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.608 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.621 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.635 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.651 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.309.661 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=4. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.310.267 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=4.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.310.305 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.310.324 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.310.339 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.310.350 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.310.366 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.310.379 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=5. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.311.003 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=5.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.311.036 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.311.054 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.311.066 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.311.078 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.311.095 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.311.106 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=6. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.312.145 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=6.) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.312.187 [memory.cpp:559]619758 aclrtDeviceEnablePeerAccess: start to execute aclrtDeviceEnablePeerAccess [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.312.206 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.312.219 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=7. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.312.232 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.312.248 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.312.267 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=7. [INFO] DRV(619427,python3.9):2025-04-03-09:35:45.313.355 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=7.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.394 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.480 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.539 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.584 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.618 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.637 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.657 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.672 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalPureMatmul_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.690 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.730 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.751 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.767 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.794 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.809 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.826 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.840 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulAllReduce_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.856 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.892 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.917 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.934 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.949 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.962 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.980 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.313.996 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.019 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.066 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.093 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.109 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.125 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.140 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.160 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.177 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmul_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.193 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.206 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.233 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.252 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.268 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.283 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.306 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.323 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulV2_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.339 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.376 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.396 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.411 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.427 [runtime.cc:1664] 619758 MallocProgramAndReg: MAGIC_ELF new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.440 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.468 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.485 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=0, funcMode=0, kernelInfoExt=LcalAllGatherMatmulReduceScatter_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.502 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.534 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.557 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.573 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.595 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.611 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.633 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.657 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.669 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.681 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.696 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.709 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.734 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.747 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.761 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.777 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.794 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.819 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.831 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.844 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.861 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.879 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.904 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.918 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.930 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.943 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.959 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.983 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.314.995 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.008 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.024 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.047 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.068 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.080 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllReduce_int64_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.092 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllReduce_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.755 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.801 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.845 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.859 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.871 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.886 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.907 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.943 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.956 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.969 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.315.985 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.006 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.037 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.048 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.061 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.076 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.098 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.136 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.150 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.316.174 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.316.178 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.191 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.316.202 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.316.210 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.234 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.241 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.246 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.255 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.260 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.265 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.316.276 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.316.278 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.316.289 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.299 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.316.299 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.308 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.315 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.321 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.330 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.346 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.367 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.403 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.416 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.429 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.443 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.460 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.497 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.512 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAllGather_int64_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.532 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAllGather_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.546 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.569 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.605 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.618 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.631 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.645 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.660 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.695 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.708 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.721 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.737 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.757 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.796 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.810 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.824 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.838 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.854 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.893 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.905 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.919 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.932 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.953 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.316.992 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.004 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.018 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.039 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.055 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.094 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.108 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.123 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.137 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.159 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.197 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.211 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalReduceScatter_int64_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.227 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalReduceScatter_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.241 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.258 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.298 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.310 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.323 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_int, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.338 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.358 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.396 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.407 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.419 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.436 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.453 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.493 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.504 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.517 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.532 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.548 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.596 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.609 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.623 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.636 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.651 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.687 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.699 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_float16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.712 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_float16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.726 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.743 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.784 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.795 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.810 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.822 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.841 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.877 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.890 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalAll2All_int64_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.902 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalAll2All_int64_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.917 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.933 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.971 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.985 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=LcalBroadcast. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.317.997 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=LcalBroadcast, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.318.057 [api_impl.cc:1990] 619758 DevMalloc: size=423624736, type=1024, moduleId=33. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.318.372 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=423624736, type=1024, moduleId=33, start ptr=0x12c041200000, end ptr=0x12c05a600020 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.318.407 [api_impl.cc:2664] 619758 MemSetSync: destMax=423624704, value=0, count=423624704 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.365 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619419, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.407 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619423, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.428 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619431, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.449 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619435, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.466 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619439, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.481 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619443, num=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.500 [npu_driver.cc:3571] 619758 SetIpcMemPid: set ipc mem pid success, name=000973a300000000000003180200ffc80002, pid[0]=619447, num=1 [2025-04-03 09:35:44.635452] [debug] [619427] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend310p success [2025-04-03 09:35:44.635464] [debug] [619427] [op_register.h:90] Mki register kernel binary MemsetKernel-ascend910b success [2025-04-03 09:35:45.151721] [warn] [619758] [env.h:29] env LCAL_COMM_ID is too long or not exist! [2025-04-03 09:35:45.151748] [debug] [619758] [lcal_sock_exchange.cpp:140] curRank: 2 serverRank: 0 ip: 127.0.0.1 port: 10067 [2025-04-03 09:35:45.151756] [debug] [619758] [lcal_sock_exchange.cpp:272] Client side 2 begin to connect [2025-04-03 09:35:45.151950] [debug] [619758] [lcal_sock_exchange.cpp:93] rank:2 UUID 80f2326c-7eb3-40cd-b189-6f4e43d0acc5 [2025-04-03 09:35:45.308264] [debug] [619758] [lcal_comm.cpp:386] GetDev : localRankSize_ : 8 localRank_: 2 rank :2 rankSize :8 [2025-04-03 09:35:45.308474] [debug] [619758] [lcal_comm.cpp:406] rank 2 devId: 2, otherDevList : 0, 1, 2, 3, 4, 5, 6, 7 [2025-04-03 09:35:45.308485] [info] [619758] [lcal_comm.cpp:407] AllGather: Get other rank dev id success [2025-04-03 09:35:45.308492] [info] [619758] [lcal_comm.cpp:242] rank 2/8 running devId:2 [2025-04-03 09:35:45.308541] [debug] [619758] [lcal_comm.cpp:91] rtGetSocVersion -- The result after converting ver to string is:Ascend910B3 [2025-04-03 09:35:45.308583] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 0, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.308728] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 1, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309007] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 3, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.309582] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 4, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.310296] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 5, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.311029] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 6, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.312176] [debug] [619758] [lcal_comm.cpp:340] 2 <-----> 7, halGetPairDevicesInfo: *value = 0 [2025-04-03 09:35:45.313374] [debug] [619758] [lcal_comm.cpp:372] EnablePeerAccess succeed2 [2025-04-03 09:35:45.318014] [debug] [619758] [lcal_comm.cpp:249] Prepare to InitCommMem localRankSize_ -> 8, localRank_ -> 2 [2025-04-03 09:35:45.318042] [debug] [619758] [lcal_comm.cpp:447] maxBuffSize 423624704 [2025-04-03 09:35:45.318394] [debug] [619758] [lcal_comm.cpp:455] peerMem[rank2], allocate finished. [2025-04-03 09:35:45.340222] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 0 pid[0]: 619419 [2025-04-03 09:35:45.340255] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 1 pid[1]: 619423 [2025-04-03 09:35:45.340264] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 2 pid[2]: 619427 [2025-04-03 09:35:45.340271] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 3 pid[3]: 619431 [2025-04-03 09:35:45.340276] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 4 pid[4]: 619435 [2025-04-03 09:35:45.340282] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 5 pid[5]: 619439 [2025-04-03 09:35:45.340290] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 6 pid[6]: 619443 [2025-04-03 09:35:45.340297] [debug] [619758] [lcal_comm.cpp:472] rank : 2, otherRank : 7 pid[7]: 619447 [2025-04-03 09:35:45.340303] [debug] [619758] [lcal_comm.cpp:474] AllGather: Get other rank pid [2025-04-03 09:35:45.340514] [debug] [619758] [lcal_comm.cpp:557] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340772] [debug] [619758] [lcal_comm.cpp:515] rank 0 mem name: 0009739b000000000000031b0000ffc00000 [2025-04-03 09:35:45.340782] [debug] [619758] [lcal_comm.cpp:515] rank 1 mem name: 0009739f00000000000003160100ffc40001 [2025-04-03 09:35:45.340788] [debug] [619758] [lcal_comm.cpp:515] rank 2 mem name: 000973a300000000000003180200ffc80002 [2025-04-03 09:35:45.340792] [debug] [619758] [lcal_comm.cpp:515] rank 3 mem name: 000973a700000000000003150300ffcc0003 [2025-04-03 09:35:45.340809] [debug] [619758] [lcal_comm.cpp:515] rank[INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.340.868 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.341.999 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=0009739b000000000000031b0000ffc00000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.342.019 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.342.897 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=0009739f00000000000003160100ffc40001. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.342.917 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.343.836 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=000973a700000000000003150300ffcc0003. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.343.855 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.357.961 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=000973ab00000000000003170400ffd00004. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.357.982 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.359.051 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=000973af000000000000031c0500ffd40005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.359.070 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.359.938 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=000973b300000000000003190600ffd80006. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.359.956 [api_impl.cc:4286] 619758 IpcOpenMemory: open ipc memory. name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.559 [npu_driver.cc:3195] 619758 OpenIpcMem: Open ipc mem success, name=000973b7000000000000031a0700ffdc0007. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.602 [api_impl.cc:1990] 619758 DevMalloc: size=132576, type=1024, moduleId=33. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.634 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=132576, type=1024, moduleId=33, start ptr=0x12c0c0016000, end ptr=0x12c0c00365e0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.360.655 [memory.cpp:421]619758 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 132520, srcSize = 132520, kind = 1 [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:45.360.850 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:38] Initialize] Successfully initialize LCCL group hccl_world_group [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.360.904 [device.cpp:143]619758 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.929 [api_impl.cc:2999] 619758 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.945 [device.cc:306] 619758 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.959 [runtime.cc:3393] 619758 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.976 [runtime.cc:3560] 619758 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.360.991 [api_impl.cc:2987] 619758 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.361.004 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.361.021 [api_impl.cc:2992] 619758 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.361.039 [api_impl.cc:3025] 619758 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.361.056 [acl.cpp:115]619758 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.361.073 [acl.cpp:121]619758 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.361.092 [device.cpp:154]619758 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [WARNING] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:45.361.127 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:146] InitializeByRootInfoConfig] Start to initialize communicator by HcclCommInitRootInfo for hccl_world_group, hcclBufferSize is 200 MB. hcclDeterministic is 0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.361.158 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.177 [adapter_rts.cc:198] [619758][hrtGetDeviceRefresh]deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.209 [externalinput.cc:2065] [619758]HCCL_LOG_CONFIG set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.227 [externalinput.cc:490] [619758]HCCL_CONNECT_TIMEOUT set by environment to [600]s [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.240 [externalinput.cc:429] [619758]HCCL_EXEC_TIMEOUT set by default to [1836]s [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.254 [externalinput.cc:543] [619758]HCCL_INTRA_PCIE_ENABLE set by default to [1], HCCL_INTRA_ROCE_ENABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.269 [externalinput.cc:622] [619758]environmental variable PROFILING_MODE and GE profiling option is not set, default: false [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.283 [externalinput.cc:714] [619758]HCCL_WHITELIST_DISABLE set by environment to [1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.297 [externalinput.cc:821] [619758]HCCL_IF_IP is not set [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.307 [externalinput.cc:877] [619758]HCCL_SOCKET_IFNAME set by default to [EmptyString] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.319 [externalinput.cc:844] [619758]HCCL_SOCKET_FAMILY is not set and is used by default [AF_INET] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.332 [externalinput.cc:728] [619758]HCCL_DEVICE_NIC_DISABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.357 [externalinput.cc:758] [619758]HCCL_HOST_RDMA_ENABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.370 [externalinput.cc:806] [619758]HCCL_IF_BASE_PORT set by default to [60000] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.382 [externalinput.cc:1508] [619758]HCCL_ALGO is not set [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.395 [externalinput.cc:1671] [619758][CannVersion][Verification]environmental variable LD_LIBRARY_PATH is too long, actual len[2016], maxlen[1024] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.409 [externalinput.cc:462] [619758]HCCL_HIGH_PERF_ENABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.423 [externalinput.cc:1519] [619758]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.435 [externalinput.cc:1554] [619758]HCCL_RDMA_SL set by default to [4] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.361.579 [api_c_soc.cc:70] 619758 rtGetSocVersion: soc version is Ascend910B3 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.600 [externalinput.cc:1599] [619758]HCCL_RDMA_TIMEOUT set by default to [20] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.620 [externalinput.cc:1633] [619758]HCCL_RDMA_RETRY_CNT set by default to [7] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.633 [externalinput.cc:1778] [619758]HCCL_BUFFSIZE set by default to [200]M [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.645 [externalinput.cc:1807] [619758]HCCL_OP_BASE_FFTS_MODE_ENABLE set by default to [true] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.659 [externalinput.cc:515] [619758]HCCL_DETERMINISTIC set by default to [false] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.671 [externalinput.cc:1841] [619758]HCCL_DIAGNOSE_ENABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.682 [externalinput.cc:1929] [619758]HCCL_ENTRY_LOG_ENABLE set by default to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.694 [externalinput.cc:1949] [619758]HCCL_INTER_HCCS_DISABLE is not set, default value is 0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.707 [externalinput.cc:1969] [619758]HCCL_INTER_VNIC_DISABLE is not set, default value is 0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.729 [externalinput.cc:2005] [619758]HCCL_OP_EXPANSION_MODE is not set, aicpuUnfold is [0], aivMode is [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.744 [externalinput.cc:2036] [619758]environmental variable ASCEND_MC2_DEBUG_MODE is not set [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.758 [externalinput.cc:2209] [619758]HCCL_CONCURRENT_ENABLE is not set, default value is [0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.771 [externalinput.cc:2178] [619758][ParseRetryEnable] HCCL_OP_RETRY_ENABLE is not set. The retryEnable of all levels is set to false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.783 [externalinput.cc:889] [619758]HCCL_SEND_CQ_DEPTH set by default to [8192] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.798 [externalinput.cc:914] [619758]HCCL_DFT is not set, default value is 1. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.812 [externalinput.cc:2240] [619758]HCCL_OP_COUNTER_ENABLE set by default to [1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.829 [externalinput.cc:2308] [619758]HCCL_OP_RETRY_PARAMS is not set, default value MaxCnt is 1, HoldTime is 5000, IntervalTime is 1000 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.844 [externalinput.cc:2339] [619758]HCCL_LOGIC_SUPERPOD_ID is not set, default value[] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.856 [externalinput.cc:2296] [619758]HCCL_STUCK_DETECT_TIME is set default [612]s. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.868 [externalinput.cc:407] [619758]HCCL_RDMA_PCIE_DIRECT_POST_NOSTRICT set by default to [EmptyString], rdmaFastPost is [0] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:45.361.884 [externalinput.cc:672] [619758][Parse][MultiQpSrcPortConfigPath]environmental variable HCCL_RDMA_QP_PORT_CONFIG_PATH is empty,but HCCL_RDMA_QP_PORT_CONFIG_PATH is not set [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:45.361.917 [env_config.cc:127] [619758]HCCL_HOST_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:45.361.931 [env_config.cc:131] [619758]HCCL_NPU_SOCKET_PORT_RANGE is not set. Multi-process will not be supported! [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.944 [env_config.cc:302] [619758]HCCL_RDMA_TC set by default to [132] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.955 [env_config.cc:302] [619758]HCCL_RDMA_SL set by default to [4] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.969 [op_base.cc:1091] [619758]Entry-HcclCommInitRootInfoInner:ranks[8], rank[2], rootinfo: host ip[192.168.0.44] port[60008] nicDeploy[1] identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.361.988 [op_base.cc:128] [619758][HcclGetDeviceId] deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.362.002 [op_base.cc:144] [619758][GetHcclOpInfoCtx] Set device, use g_hcclDeviceId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.362.029 [rank_consistentcy_checker.cc:32] [619758][GetInstance] get deviceLogicId[2] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.362.068 [api_impl.cc:3100] 619758 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.362.084 [api_impl.cc:3111] 619758 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.362.096 [api_impl.cc:3130] 619758 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.362.319 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.362.353 [network_manager.cc:169] [619758]NetworkManager: devicePhyId[2], devicePhyId_[2] deviceLogicId_[2], hasBackup[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.394 [ra_host.c:1594]tid:619758,ra_socket_set_white_list_status(1594) : Input parameters: enable[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.362.433 [adapter_hccp.cc:1450] [619758]set host socket whitelist status[0] success. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.466 [ra_host.c:205]tid:619758,ra_init(205) : Input parameters: phy_id[2], nic_position:[0] hdc_type:[6] [INFO] ROCE(619427,python3.9):2025-04-03-09:35:45.362.515 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.529 [ra_peer.c:776]tid:619758,ra_peer_init(776) : [init][ra_peer]ra_peer_init phy_id[2] start [INFO] ROCE(619427,python3.9):2025-04-03-09:35:45.362.598 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] ROCE(619427,python3.9):2025-04-03-09:35:45.362.670 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.698 [rs.c:327]tid:619758,rs_pthread_mutex_init(327) : mutex init ok [WARNING] ROCE(619427,python3.9):2025-04-03-09:35:45.362.865 [network_comm.c:155]get_saved_tls_config_file_path(155) : conf_path[/home/jenkins/cert] is invalid, reason[-2]. [WARNING] ROCE(619427,python3.9):2025-04-03-09:35:45.362.894 [network_comm.c:166]get_saved_tls_config_file_path(166) : conf_path[/home/HwDmUser/cert] is invalid, reason[-2]. [WARNING] HCCP(619427,python3.9):2025-04-03-09:35:45.362.908 [rs_ssl.c:1513]tid:619758,rs_ssl_init(1513) : can not get manage data, ret(-2). maybe not set tls [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.919 [rs_ssl.c:1529]tid:619758,rs_ssl_init(1529) : TLS SWITCH (0) [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.362.965 [rs_epoll.c:692]tid:619758,rs_epoll_connect_handle_init(692) : rs_create_epoll ok [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.099 [rs_epoll.c:487]tid:619772,rs_epoll_handle(487) : thread begin! thread_id:281469221728160, pid:619427, ppid:619411 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.204 [rs_epoll.c:713]tid:619758,rs_epoll_connect_handle_init(713) : RS INIT OK! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.216 [rs_epoll.c:623]tid:619776,rs_connect_handle(623) : thread begin! thread_id:281468802690976, pid:619427, ppid:619411 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.226 [rs.c:366]tid:619758,rs_init_rscb_cfg(366) : rs_epoll_connect_handle_init ok cost [0.293000] ms [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.338 [rs_epoll.c:641]tid:619776,rs_connect_handle(641) : pthread[connect_pthread] is alive! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.350 [rs_epoll.c:415]tid:619772,bind_data_cpu(415) : host not need bind cpu, info[1] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.363.366 [rs_epoll.c:511]tid:619772,rs_epoll_handle(511) : pthread[epoll_pthread] is alive! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.235 [rs.c:471]tid:619758,rs_init(471) : rs init success, chip_id[2] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.272 [ra_peer.c:813]tid:619758,ra_peer_init(813) : [init][ra_peer]ra_peer_init phy_id[2] succ [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.370.292 [adapter_hccp.cc:583] [619758]init ra success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.370.304 [network_manager.cc:222] [619758]NetworkManager nicDeploy[0] deviceLogicId_[2] devicePhyId_[2] init ra OK, nicSocketMap size[0], isHostUseDevNic[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.338 [ra_host.c:1764]tid:619758,ra_get_ifnum(1764) : Input parameters: phy_id[2], nic_position:[0], is_all:[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.356 [ra_peer.c:859]tid:619758,ra_peer_get_ifnum(859) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[2] start [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.383 [ra_peer.c:867]tid:619758,ra_peer_get_ifnum(867) : [get][ra_peer_ifnum]ra_peer_get_ifnum phy_id[2] succ [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.370.395 [adapter_hccp.cc:1378] [619758][Get][HostIf]hrtGetIfNum success. ifAddrNum[6]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.412 [ra_host.c:1814]tid:619758,ra_get_ifaddrs(1814) : Input parameters: phy_id[2], nic_position:[0], is_all[0], interface num[6] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.427 [ra_peer.c:874]tid:619758,ra_peer_get_ifaddrs(874) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[2] start [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.439 [rs_socket.c:2743]tid:619758,rs_peer_fill_ifaddr_infos(2743) : ifname[lo] addr[0x0100007f] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.454 [rs_socket.c:2743]tid:619758,rs_peer_fill_ifaddr_infos(2743) : ifname[enp67s0f5] addr[0x2c00a8c0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.465 [rs_socket.c:2743]tid:619758,rs_peer_fill_ifaddr_infos(2743) : ifname[docker0] addr[0x010011ac] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.478 [rs_socket.c:2749]tid:619758,rs_peer_fill_ifaddr_infos(2749) : ifname[lo] scope_id[0] flowinfo[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.491 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.500 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.513 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.525 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.539 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.557 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.572 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.585 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.612 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.625 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.638 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.651 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.662 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.674 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.687 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.698 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x01 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.711 [rs_socket.c:2749]tid:619758,rs_peer_fill_ifaddr_infos(2749) : ifname[enp67s0f5] scope_id[2] flowinfo[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.722 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.734 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.744 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.756 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.769 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.779 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.790 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.802 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.812 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0x8b [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.823 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0x0e [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.836 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0xc4 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.847 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0x02 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.858 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x33 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.873 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0x1c [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.886 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0x33 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.898 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0x2f [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.911 [rs_socket.c:2749]tid:619758,rs_peer_fill_ifaddr_infos(2749) : ifname[endvnic] scope_id[7] flowinfo[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.922 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[0] 0xfe [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.933 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[1] 0x80 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.945 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[2] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.959 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[3] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.370.991 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[4] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.004 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[5] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.017 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[6] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.028 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[7] 0x00 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.043 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[8] 0xed [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.053 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[9] 0xaf [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.065 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[10] 0x6d [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.076 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[11] 0xc3 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.088 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[12] 0x6f [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.100 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[13] 0xad [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.114 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[14] 0xb4 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.127 [rs_socket.c:2751]tid:619758,rs_peer_fill_ifaddr_infos(2751) : addr[15] 0xb2 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.139 [ra_peer.c:882]tid:619758,ra_peer_get_ifaddrs(882) : [get][ra_peer_ifaddrs] ra_peer_get_ifaddrs phy_id[2] succ [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.156 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[0] ifname[lo] ip[127.0.0.1%lo] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.175 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[1] ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.190 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[2] ifname[docker0] ip[172.17.0.1%docker0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.204 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[3] ifname[lo] ip[::1%lo] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.219 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[4] ifname[enp67s0f5] ip[fe80::8b0e:c402:331c:332f%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.233 [adapter_hccp.cc:1407] [619758][Get][HostIf]hrtGetIfAddress: idx[5] ifname[endvnic] ip[fe80::edaf:6dc3:6fad:b4b2%endvnic] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.250 [sal.cc:365] [619758]get host ip success. host ifname[enp67s0f5] ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.265 [sal.cc:432] [619758]nic class[normal]: find nic[192.168.0.44%enp67s0f5] success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.280 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:45.371.302 [ra_host.c:340]tid:619758,ra_socket_init_v1(340) : socket init:mode=0 phy_id=2 scope_id=0 family=2 ip=192.168.0.44 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.317 [adapter_hccp.cc:818] [619758]socket init v1 success, socketHandle[0xfffe880faba0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.330 [network_manager.cc:1218] [619758]ip[192.168.0.44%enp67s0f5] socket init OK [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.344 [network_manager.cc:1257] [619758]ip[192.168.0.44%enp67s0f5] socket start success socketHandle[0xfffe880faba0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.390 [adapter_hal.cc:298] [619758]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.413 [sal.cc:593] [619758]IsHostUseDevNic[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.433 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.446 [network_manager.cc:169] [619758]NetworkManager: devicePhyId[4294967295], devicePhyId_[2] deviceLogicId_[2], hasBackup[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.488 [tsd_client.cpp:189][TsdCapabilityGet][tid:619758] TsdCapabilityGet Begin. [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.506 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.519 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.534 [process_mode_manager.cpp:1042][CapabilityGet][tid:619758] [ProcessModeManager] enter into CapabilityGet process deviceId[2] type[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.548 [adapter_tdt.cc:66] [619758]Get TsdCapability success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.560 [network_manager.cc:175] [619758][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:45.371.578 [network_manager.cc:99] [619758][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.597 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.608 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.620 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619758] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.631 [process_mode_manager.cpp:386][InitTsdClient][tid:619758] [TsdClient] tsd client has been inited before [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.651 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619758] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619427,python3.9):2025-04-03-09:35:45.371.673 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619758] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.690 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619758] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.703 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619758] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.716 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619758] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.741 [version_verify.cpp:114][SpecialFeatureCheck][tid:619758] VersionVerify: previous type[48], supported [INFO] TDT(619427,python3.9):2025-04-03-09:35:45.371.778 [hdc_client.cpp:154][TsdRecvData][tid:619758] recv timeout is:150000 ms [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.186 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.188 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.189 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.191 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.191 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.476.191 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.305 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.323 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.337 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.344 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.349 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.357 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.362 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.371 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.375 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.382 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.386 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.394 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.400 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.405 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.413 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.419 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.423 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.431 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.434 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.459 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.462 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.465 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.471 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.473 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.483 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.486 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.494 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.496 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.499 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.508 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.510 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.521 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.523 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.476.534 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.545 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.476.556 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.828.174 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.828.176 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.828.200 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:45.828.205 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.213 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.218 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.226 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.231 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.828.252 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.828.255 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.828.265 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:45.828.270 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.280 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:45.828.286 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.188.673 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:46.188.701 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:46.188.714 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:46.188.725 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.172 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.172 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.174 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.177 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.179 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.308.179 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.263 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.277 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.289 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.293 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.301 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.306 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.311 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.324 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.327 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.329 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.339 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.342 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.350 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.354 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.357 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.362 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.366 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.372 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.374 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.386 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.388 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.391 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.399 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.407 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.410 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.421 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.421 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.424 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.432 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.434 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.443 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.445 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.308.453 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.472 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.476 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.308.486 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.340.169 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.340.169 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.340.192 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.340.197 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.205 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.208 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.218 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.222 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.340.227 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.340.233 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.340.239 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.340.244 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.251 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.340.256 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.852.179 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.852.179 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.852.213 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:46.852.221 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.233 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.237 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.247 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.249 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.852.273 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.852.273 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.852.287 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:46.852.288 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.303 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:46.852.305 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.167 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.173 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.171 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.179 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.180 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.140.179 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.266 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.281 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.289 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.299 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.302 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.309 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.313 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.320 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.323 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.324 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.331 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.334 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.350 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.358 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.362 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.367 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.372 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.379 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.383 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.387 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.392 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.397 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.401 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.404 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.408 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.414 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.416 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.418 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.424 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.434 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.436 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.449 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.451 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.140.460 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.470 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.140.481 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.364.167 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.364.172 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.364.190 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.364.196 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.205 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.207 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.216 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.221 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.364.226 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.364.232 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.364.238 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.364.244 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.250 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.364.255 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.732.173 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.732.209 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619758] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.732.230 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619758] [TsdClient] DeviceMsgProc recvMsg realDeviceId[2] msgType[49] localDevId[2] rspCode[0]heterogeneousSubPid[6148], tsdSupportLevel_[63] [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.732.244 [hdc_client.cpp:172][TsdRecvData][tid:619758] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.732.256 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619758] OpenSubProc success stype:0, pid:6148 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.732.257 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.732.268 [adapter_tdt.cc:39] [619758]Open TsdProcess success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.732.279 [network_manager.cc:118] [619758][TsdProcessOpen]hrtTsdProcessOpen success, subPid[6148], devicePhyId_[2], deviceLogicId_[2], hasBackup[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.732.295 [network_manager.cc:180] [619758][NetworkManager][Init]open tsd by process success, devicePhyId[2], deviceLogicId_[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.732.349 [ra_host.c:205]tid:619758,ra_init(205) : Input parameters: phy_id[2], nic_position:[1] hdc_type:[18] [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.732.364 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] DRV(619427,python3.9):2025-04-03-09:35:47.732.397 [devdrv_hccl.c:103][ascend][curpid:619427,619758][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619427) [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.732.412 [dl_hal_function.c:218]dl_hal_init(218) : dl_hal_init success, no need to dlopen libascend_hal.so! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.732.425 [ra_hdc.c:580]tid:619758,ra_hdc_init(580) : hdc init start! logic id is 2, phy id is 2, hdc_type is 18 [INFO] DRV(619427,python3.9):2025-04-03-09:35:47.732.658 [hdc_core.c:1982][ascend][curpid:619427,619758][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4269; pid=619427) [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.732.897 [ra_hdc.c:616]tid:619758,ra_hdc_init(616) : hdc init OK! phy_id[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.736.319 [adapter_hccp.cc:583] [619758]init ra success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.736.339 [network_manager.cc:222] [619758]NetworkManager nicDeploy[1] deviceLogicId_[2] devicePhyId_[2] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.736.367 [topoinfo_detect.cc:534] [619758]NetworkManager start host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.736.388 [ra_host.c:1764]tid:619758,ra_get_ifnum(1764) : Input parameters: phy_id[2], nic_position:[1], is_all:[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.736.769 [adapter_hccp.cc:1538] [619758][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.736.785 [ra_host.c:1814]tid:619758,ra_get_ifaddrs(1814) : Input parameters: phy_id[2], nic_position:[1], is_all[0], interface num[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.130 [adapter_hccp.cc:1568] [619758][Get][DeviceIP]hrtGetIfAddress: idx[0] ifname[eth2] ip[29.105.165.16%eth2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.148 [adapter_hccp.cc:1568] [619758][Get][DeviceIP]hrtGetIfAddress: idx[1] ifname[eth2] ip[fe80::2287:ecff:fe02:4625%eth2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.162 [topoinfo_detect.cc:577] [619758]select AF_INET family as device socket family. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.177 [topoinfo_detect.cc:756] [619758]no backup device ip: use 0 as device ip. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.187 [topoinfo_detect.cc:294] [619758][HCCL_TRACE]SetupAgent rankNum[8], rank[2], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], serverPort[60008]deviceType[2], logicDevId[2], phydevId[2], deviceIp[29.105.165.16] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.217 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.232 [hccl_socket.cc:516] [619758][Get][NicHandleInfo]phyId[2], nicSocketMap[1] localIp[[192.168.0.44%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.246 [hccl_socket.cc:217] [619758][Connect] localIp[192.168.0.44%enp67s0f5], remoteIp[192.168.0.44], socket_handle[281468669504416], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], port[60008] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.737.258 [adapter_hccp.cc:1050] [619758]batch conncet, port[60008] remoteip[7f7f7f7f] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.737.271 [ra_host.c:759]tid:619758,ra_socket_batch_connect(759) : Input parameters: [0]th, phy_id[2], local_ip[192.168.0.44], remote_ip[192.168.0.44], port[60008], tag[topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008], cnt[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.737.296 [rs_socket.c:1810]tid:619758,rs_socket_IP_convert(1810) : local IP[0xfffe2c00a8c0], ret:-1, remote IP[0xfffe2c00a8c0], ret:-1 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.737.309 [rs_socket.c:1819]tid:619758,rs_socket_IP_convert(1819) : local IP[192.168.0.44], ret:0, remote IP[192.168.0.44], ret:-1 [WARNING] HCCP(619427,python3.9):2025-04-03-09:35:47.737.329 [rs_socket.c:132]tid:619758,rs_get_conn_info(132) : conn node for IP(192.168.0.44) server_port(60008) tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008) not found [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.737.343 [rs_socket.c:1880]tid:619758,rs_socket_batch_connect(1880) : create conn node for {remote_ip(192.168.0.44), server_port(60008), tag(topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008)}! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.737.998 [rs_socket.c:1577]tid:619776,rs_connect_bind_client(1577) : client bind success. client family 2 addr 192.168.0.44:48651, fd:53 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.738.029 [rs_drv_socket.c:59]tid:619776,rs_drv_connect(59) : IP(192.168.0.44) port 60008 family 2 fd:53 begin [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.738.114 [rs_drv_socket.c:98]tid:619776,rs_drv_connect(98) : client connect success. client family 2 addr 192.168.0.44:48651, server addr 192.168.0.44:60008, fd:53 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.738.133 [rs_socket.c:1427]tid:619776,rs_conn_cost_time(1427) : socket [53] connect success! cost [0.791000] ms [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.738.170 [rs_socket.c:1392]tid:619776,rs_socket_tag_sync(1392) : [client]send tag success! ret:256, tag_sync_time:1, client_ip:192.168.0.44 server_ip:192.168.0.44 server_port:60008 tag:topo_detect_default_tag_192.168.0.44%enp67s0f5_60008_0_1743644144960128_60008 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.738.424 [topoinfo_exchange_agent.cc:212] [619758]TopoInfoExchangeAgent get socket success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.738.475 [topoinfo_exchange_agent.cc:236] [619758]local rank[2] get socket connection with server[192.168.0.44] port[60008] success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.738.491 [topoinfo_exchange_agent.cc:43] [619758]TopoExchangeAgent: client connect with server ip[192.168.0.44] port[60008] success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.738.505 [topoinfo_exchange_agent.cc:345] [619758]ConstructRankTableServerId serverId 192.168.0.44 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.738.591 [topoinfo_exchange_agent.cc:77] [619758]topo exchange client send rank basic info success. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.800.773 [api_impl.cc:2096] 619758 HostMalloc: size=2485 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.800.819 [api_error.cc:1260] 619758 HostMalloc: Host memory malloc succeed,size=2485 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.801.109 [api_error.cc:1278] 619758 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.129 [topoinfo_exchange_agent.cc:80] [619758]topo exchange client get rank basic info success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.185 [topoinfo_detect.cc:303] [619758]topo detect completed. myrank[2], totalranks[8], myhost[192.168.0.44%enp67s0f5], totalservers[1]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.216 [config.cc:198] [619758]rank_table: "Unique deviceNum":"8", "Unique serverNum":"1", "para_plane_nic_location":"1", "para_plane_nic_num":"0";"para_plane_nic_name":[] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.236 [config.cc:210] [619758]"device_information":{rankID[0],serverId[192.168.0.44],deviceId[0],deviceType[2],deviceIp[192.168.100.101],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.251 [config.cc:210] [619758]{rankID[1],serverId[192.168.0.44],deviceId[1],deviceType[2],deviceIp[29.105.19.202],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.264 [config.cc:210] [619758]{rankID[2],serverId[192.168.0.44],deviceId[2],deviceType[2],deviceIp[29.105.165.16],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.276 [config.cc:210] [619758]{rankID[3],serverId[192.168.0.44],deviceId[3],deviceType[2],deviceIp[29.105.51.138],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.295 [config.cc:210] [619758]{rankID[4],serverId[192.168.0.44],deviceId[4],deviceType[2],deviceIp[29.105.24.184],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.306 [config.cc:210] [619758]{rankID[5],serverId[192.168.0.44],deviceId[5],deviceType[2],deviceIp[29.105.72.114],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.317 [config.cc:210] [619758]{rankID[6],serverId[192.168.0.44],deviceId[6],deviceType[2],deviceIp[29.105.134.38],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.328 [config.cc:210] [619758]{rankID[7],serverId[192.168.0.44],deviceId[7],deviceType[2],deviceIp[29.105.73.96],hostIp[192.168.0.44]}, [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.801.342 [adapter_hccp.cc:1067] [619758]ra socket batch close fdhandle[0xfffe88054bf0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.801.358 [ra_host.c:796]tid:619758,ra_socket_batch_close(796) : Input parameters: [0]th, phy_id[2], local_ip[192.168.0.44], cnt[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.801.376 [rs_socket.c:1937]tid:619758,rs_socket_batch_close(1937) : conn node of IP(192.168.0.44) fd:53, state:9 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.848.208 [adapter_hccp.cc:1099] [619758]ra socket batch close success,take time [46852]us [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:47.848.242 [hccl_socket.cc:232] [619758][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.848.263 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.284 [ra_host.c:404]tid:619758,ra_socket_deinit(404) : Input parameters: phy_id[2] family[2] local_ip[192.168.0.44] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.300 [rs.c:1629]tid:619758,rs_socket_deinit(1629) : rs socket deinit start, phy_id:2 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.313 [rs.c:1641]tid:619758,rs_socket_deinit(1641) : socket deinit local IP is 0x2c00a8c0, ret:-1 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.328 [rs.c:1653]tid:619758,rs_socket_deinit(1653) : socket deinit success, phy_id:2, local_ip:192.168.0.44 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.848.355 [network_manager.cc:1233] [619758]ip[192.168.0.44%enp67s0f5] deinit success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.848.370 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.382 [ra_host.c:248]tid:619758,ra_deinit(248) : Input parameters: phy_id[2], nic_position:[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.393 [ra_peer.c:821]tid:619758,ra_peer_deinit(821) : [deinit][ra_peer]ra_peer_deinit phy_id[2] start [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.411 [rs.c:1783]tid:619758,rs_deinit(1783) : epoll wait up ok, rscb->conn_flag:1 [WARNING] HCCP(619427,python3.9):2025-04-03-09:35:47.848.416 [rs_epoll.c:523]tid:619772,rs_epoll_handle(523) : SHUT DOWN event!!! eventfd:51 [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.848.443 [rs_epoll.c:544]tid:619772,rs_epoll_handle(544) : QUIT thread_id:281469221728160, pid:619427, read num:8 [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.876.171 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.876.173 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.876.191 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.876.197 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.203 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.220 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.223 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.232 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.876.236 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.876.243 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.876.248 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.876.253 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.264 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.876.266 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.948.720 [rs_epoll.c:679]tid:619776,rs_connect_handle(679) : QUIT thread_id:281468802690976, pid:619427 [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.968.763 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.968.786 [rs.c:1810]tid:619758,rs_deinit(1810) : rs_deinit chip_id[2] ok [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.968.798 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.968.809 [ra_peer.c:852]tid:619758,ra_peer_deinit(852) : [deinit][ra_peer]ra_peer_deinit phy_id[2] succ [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.968.821 [dl_hal_function.c:199]dl_hal_deinit(199) : dl_hal_deinit success, no need to dlclose libascend_hal.so! [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.968.844 [network_manager.cc:409] [619758]NetworkManager: deinit nic success, nicPosition[0] ref[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.968.859 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.968.872 [ra_host.c:248]tid:619758,ra_deinit(248) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.968.884 [ra_hdc.c:651]tid:619758,ra_hdc_deinit(651) : hdc deinit start! phy_id[2] [INFO] DRV(619427,python3.9):2025-04-03-09:35:47.969.031 [hdc_client.c:417][ascend][curpid:619427,619758][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4269) [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.969.092 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:47.969.105 [ra_hdc.c:685]tid:619758,ra_hdc_deinit(685) : hdc deinit OK! phy_id[2] [INFO] ROCE(619427,python3.9):2025-04-03-09:35:47.969.117 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.142 [tsd_client.cpp:189][TsdCapabilityGet][tid:619758] TsdCapabilityGet Begin. [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.155 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.168 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.190 [process_mode_manager.cpp:1042][CapabilityGet][tid:619758] [ProcessModeManager] enter into CapabilityGet process deviceId[2] type[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:47.969.205 [adapter_tdt.cc:66] [619758]Get TsdCapability success. deviceLogicId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.218 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.228 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.242 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619758] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.261 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619758] add close subproc:6148, proctype:0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.277 [version_verify.cpp:114][SpecialFeatureCheck][tid:619758] VersionVerify: previous type[60], supported [INFO] TDT(619427,python3.9):2025-04-03-09:35:47.969.316 [hdc_client.cpp:154][TsdRecvData][tid:619758] recv timeout is:150000 ms [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.168 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.180 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.173 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.181 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.182 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:47.972.206 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.262 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.283 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.290 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.296 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.305 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.309 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.316 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.320 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.320 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.327 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.332 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.348 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.352 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.356 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.364 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.367 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.369 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.376 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.379 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.384 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.387 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.391 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.397 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.400 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.402 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.409 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.412 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.419 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.418 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.430 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.445 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.450 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.470 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:47.972.488 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.505 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:47.972.535 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.201.782 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619758] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.201.808 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619758] [TsdClient] DeviceMsgProc recvMsg realDeviceId[2] msgType[61] localDevId[2] rspCode[0]heterogeneousSubPid[6148], tsdSupportLevel_[63] [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.201.821 [hdc_client.cpp:172][TsdRecvData][tid:619758] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.201.832 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619758] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.201.843 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619758] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.856 [adapter_tdt.cc:52] [619758]Open TsdProcess success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.866 [network_manager.cc:405] [619758][DeInit]finish CloseHccpProcess, phy_id[2], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.878 [network_manager.cc:409] [619758]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.890 [topoinfo_detect.cc:555] [619758]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.901 [topoinfo_detect.cc:321] [619758]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.916 [topoinfo_detect.cc:219] [619758]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.932 [rank_consistentcy_checker.cc:32] [619758][GetInstance] get deviceLogicId[2] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:48.201.947 [rank_consistentcy_checker.cc:144] [619758][Record][CannVersion] version information str is empty. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.958 [op_base.cc:2231] [619758]rank table is null, rankTableCrc is 0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.984 [op_base.cc:2219] [619758]params.id.internal [hccl-619427-0-1743644148-hccl_world_group] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.201.998 [op_base.cc:993] [619758]rootInfo[hccl-619427-0-1743644148-hccl_world_group], params.logiceDevice[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.202.017 [hccl_comm.cc:98] [619758]hcclComm init workmode [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.202.029 [hccl_comm.cc:954] [619758]InitImpl Implementation isHeterogComm_[0] isHaveCpuRank_[0] deviceType[2] isSpecialType_[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.202.906 [hccl_comm.cc:1365] [619758]RegistTaskAbortHandler begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.202.926 [task_abort_handler.cc:126] [619758]TaskAbortHandler::Init commVector size is [0], ref_ count is [0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.202.940 [api_impl.cc:4192] 619758 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.202.955 [runtime.cc:4698] 619758 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.202.982 [adapter_rts.cc:42] [619758]open so: libruntime.so [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.305 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.332 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.348 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.381 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.394 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.411 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.425 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.442 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.453 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.472 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.483 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.503 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.516 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.534 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.548 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.565 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.577 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.596 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.610 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.627 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.639 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.655 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.670 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.686 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.697 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.716 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.728 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.743 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.760 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.777 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.788 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.807 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.820 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.838 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.851 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.868 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.879 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.894 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.905 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.922 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.933 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.946 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.958 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.203.971 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.203.984 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.000 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.015 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.034 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.047 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.062 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.075 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.091 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.111 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.126 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.138 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.154 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.217 [hccl_communicator_attrs.cc:407] [619758]module[0] contains [8]devices [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.228 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[0], serverId[192.168.0.44], serverIdx[0], devicePhyId[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.240 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[1], serverId[192.168.0.44], serverIdx[0], devicePhyId[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.252 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[2], serverId[192.168.0.44], serverIdx[0], devicePhyId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.263 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[3], serverId[192.168.0.44], serverIdx[0], devicePhyId[3] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.275 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[4], serverId[192.168.0.44], serverIdx[0], devicePhyId[4] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.288 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[5], serverId[192.168.0.44], serverIdx[0], devicePhyId[5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.300 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[6], serverId[192.168.0.44], serverIdx[0], devicePhyId[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.311 [hccl_communicator_attrs.cc:411] [619758]moduleIdx[0] Info: rankId[7], serverId[192.168.0.44], serverIdx[0], devicePhyId[7] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.323 [hccl_communicator_attrs.cc:425] [619758]different module contains different numbers of cards:[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.335 [hccl_communicator_attrs.cc:426] [619758]different module contains different type of cards:[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.378 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.404 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.418 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.442 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.454 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.471 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.484 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.501 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.513 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.529 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.550 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.567 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.583 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.600 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.614 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.631 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.646 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.663 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.678 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.694 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.705 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.724 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.736 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.754 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.767 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.784 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.798 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.818 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.834 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.854 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.868 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.887 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.901 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.918 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.940 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.958 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.204.970 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.204.988 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.003 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.021 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.033 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.050 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.062 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.082 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.095 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.111 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.123 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.141 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.153 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.170 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.183 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.203 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.215 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.233 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.246 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.264 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.276 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.293 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.306 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.329 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.344 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.359 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.373 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.390 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.404 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.423 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.437 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.454 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.465 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.481 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.494 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.513 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.526 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.545 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.560 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.577 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.592 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.608 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.620 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.636 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.647 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.667 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.680 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.699 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.717 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.735 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.746 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.763 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.776 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.796 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.808 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.825 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.837 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.854 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.867 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.886 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.900 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.916 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.929 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.945 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.959 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.205.975 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.205.987 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.004 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.020 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.041 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.054 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.072 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.085 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.109 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.123 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=7, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.139 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[7] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.155 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.170 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.182 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.200 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.212 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.227 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.240 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.255 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.267 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.284 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.295 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.312 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.325 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=0, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.340 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[0] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.352 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=2, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.367 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[2] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.379 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.394 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.406 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.425 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.439 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.464 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.477 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.494 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.506 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=1, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.522 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[1] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.535 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.552 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.565 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.587 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.599 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.617 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.629 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.645 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.658 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.675 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.687 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.707 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.722 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.738 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.750 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.765 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.777 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=3, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.794 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[3] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.806 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.824 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.842 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.862 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.875 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=4, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.890 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[4] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.901 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.917 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.929 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=5, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.948 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[5] otherPhyDevId[7] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.206.961 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=6, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.206.979 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[6] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.028 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[0], nicIp[192.168.100.101], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.050 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[1], nicIp[29.105.19.202], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.068 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[2], nicIp[29.105.165.16], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.085 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[3], nicIp[29.105.51.138], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.102 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[4], nicIp[29.105.24.184], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.120 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[5], nicIp[29.105.72.114], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.136 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[6], nicIp[29.105.134.38], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.152 [hccl_communicator_attrs.cc:156] [619758][Get][NicInfo]serverId[192.168.0.44], serverIdx[0], rankIndex[7], nicIp[29.105.73.96], backupNicIp[0.0.0.0], deviceNicPort[65536], deviceVnicPort[65536], backupDevicePort[65536] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.174 [hccl_communicator_attrs.cc:1021] [619758][HcclCommunicatorAttrs][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[0], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.195 [hccl_communicator.cc:1440] [619758][TcpMode][3] [1:TCP, 2:RDMA, 3:RESERVED] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.207 [hccl_communicator.cc:1450] [619758][Init][TcpMode]isTcpMode[0] nicDeploy[1] hcclDeviceNicDisable[0] hcclHostRdmaEnable[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.221 [externalinput.cc:1445] [619758]InitParsesProtocol start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.238 [externalinput.cc:1462] [619758]environmental variable HCCL_NPU_NET_PROTOCOL is not set [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:48.207.254 [externalinput.cc:1411] [619758]can't find HELPER_RES_CONFIG or HELPER_RES_FILE_PATH, get protocol type from default path [/usr/local/Ascend/latest/conf/resource.json] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.280 [externalinput.cc:1413] [619758]default resource file is not exist.RDMA is used by default. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.291 [externalinput.cc:1467] [619758]Protocol[2], remoteIsHdc[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.322 [config.cc:557] [619758][Config][SetRetryEnable]deviceType[2], superPodNum[0], serverNum[1], deviceNum[8],isDiffDeviceType[0], retryEnable[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.346 [hccl_communicator.cc:668] [619758][HcclCommunicator][SetRanksPort] devicePortSwitchOn[0], isHaveCpuRank[0], isUseRankPort[1], nicRanksPort size[8], vnicRanksPort size[0]. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.376 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.398 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.412 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=0. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.434 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=0.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.448 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.463 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.478 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.492 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=1. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.507 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=1.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.520 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.532 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.544 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.558 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=3. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.573 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=3.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.585 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.599 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.612 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.624 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=4. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.658 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=4.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.672 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.687 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.699 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.712 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=5. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.728 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=5.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.738 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.751 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.767 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.779 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=6. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.795 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=6.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.807 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.818 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.831 [api_impl.cc:3520] 619758 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.207.845 [api_impl.cc:3168] 619758 EnableP2P: Enable P2P devIdDes=2, phyIdSrc=7. [INFO] DRV(619427,python3.9):2025-04-03-09:35:48.207.861 [devdrv_pcie.c:888][ascend][curpid:619427,619758][drv][devmng][drvP2PAttrOp]Enable P2P. (logic_devid=2; peer_phy_devid=7.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.873 [p2p_mgmt.cc:74] [619758]enable p2p: local logic id:2, remote physic id:7. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.899 [adapter_hal.cc:298] [619758]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.920 [sal.cc:593] [619758]IsHostUseDevNic[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.966 [adapter_hal.cc:298] [619758]Entry-hrtDrvGetPlatformInfo [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.983 [sal.cc:593] [619758]IsHostUseDevNic[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.207.995 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.208.006 [network_manager.cc:169] [619758]NetworkManager: devicePhyId[4294967295], devicePhyId_[2] deviceLogicId_[2], hasBackup[0] [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.023 [tsd_client.cpp:189][TsdCapabilityGet][tid:619758] TsdCapabilityGet Begin. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.037 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.047 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.059 [process_mode_manager.cpp:1042][CapabilityGet][tid:619758] [ProcessModeManager] enter into CapabilityGet process deviceId[2] type[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.208.071 [adapter_tdt.cc:66] [619758]Get TsdCapability success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.208.083 [network_manager.cc:175] [619758][NetworkManager][Init]supportMultiProcHCCP[1], hasBackup[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:48.208.105 [network_manager.cc:99] [619758][TsdProcessOpen]params: halAPIVersion[467736], BACKUP_DEVICE_LOG_DEV_VERSION[467736] [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.122 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.134 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619758] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.146 [process_mode_manager.cpp:1434][ProcessOpenSubProc][tid:619758] enter into ProcessOpenSubProc subtype:0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.170 [process_mode_manager.cpp:386][InitTsdClient][tid:619758] [TsdClient] tsd client has been inited before [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.194 [process_mode_manager.cpp:1363][SetCommonOpenParamList][tid:619758] cnt:0, extra parameters:--hdcType=18, len:12 [WARNING] TDT(619427,python3.9):2025-04-03-09:35:48.208.211 [env_internal_api.cpp:22][GetEnvFromMmSys][tid:619758] Get env[ASCEND_LATEST_INSTALL_PATH] failed [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.224 [process_mode_manager.cpp:1680][GetAscendLatestIntallPath][tid:619758] latest install path:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.240 [process_mode_manager.cpp:1383][ConstructCommonOpenMsg][tid:619758] runtimePkgPath:/usr/local/Ascend/ascend-toolkit/latest [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.258 [process_mode_manager.cpp:1389][ConstructCommonOpenMsg][tid:619758] hostLibPath:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/lib/plugin/ascend:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/lib:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/examples:/usr/local/Ascend/nnal/atb/latest/atb/cxx_abi_0/tests/atbopstest:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64:/usr/local/Ascend/ascend-toolkit/latest/tools/aml/lib64/plugin:/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/ [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.208.304 [hdc_client.cpp:154][TsdRecvData][tid:619758] recv timeout is:150000 ms [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.388.174 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.388.173 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.388.199 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.388.207 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.213 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.221 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.226 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.235 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.388.240 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.388.262 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.388.261 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.275 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.388.282 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.388.298 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.174 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.177 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.178 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.178 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.181 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.804.186 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.282 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.302 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.313 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.313 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.325 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.330 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.336 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.342 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.347 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.353 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.356 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.363 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.366 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.384 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.387 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.390 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.398 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.404 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.407 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.415 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.417 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.427 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.426 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.440 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.444 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.451 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.454 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.461 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.467 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.470 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.493 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.500 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.514 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.804.531 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.548 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.804.561 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.900.170 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.900.180 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.900.194 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:48.900.211 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.218 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.227 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.231 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.241 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.900.246 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.900.255 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.900.257 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:48.900.268 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.274 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.900.284 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:48.940.706 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.940.728 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.940.747 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:48.940.760 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.412.167 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.412.172 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.412.192 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.412.205 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.208 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.220 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.224 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.234 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.412.238 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.412.262 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.412.264 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.412.277 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.282 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.412.293 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.178 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.179 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.180 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.181 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.183 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.636.184 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.266 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.279 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.288 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.297 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.306 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.305 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.317 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.329 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.343 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.347 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.361 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.369 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.387 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.387 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.397 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.403 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.414 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.422 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.427 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.434 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.441 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.446 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.458 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.468 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.471 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.476 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.481 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.489 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.490 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.494 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.500 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.503 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.636.506 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.514 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.517 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.636.527 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.924.172 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.924.179 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.924.210 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:49.924.216 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.224 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.231 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.237 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.243 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.924.251 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.924.256 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.924.265 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:49.924.268 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.279 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:49.924.282 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.311.163 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:50.311.164 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619758] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:50.311.195 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619758] [TsdClient] DeviceMsgProc recvMsg realDeviceId[2] msgType[49] localDevId[2] rspCode[0]heterogeneousSubPid[6167], tsdSupportLevel_[63] [INFO] TDT(619427,python3.9):2025-04-03-09:35:50.311.210 [hdc_client.cpp:172][TsdRecvData][tid:619758] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:50.311.224 [process_mode_manager.cpp:1461][ProcessOpenSubProc][tid:619758] OpenSubProc success stype:0, pid:6167 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.311.236 [adapter_tdt.cc:39] [619758]Open TsdProcess success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.311.247 [network_manager.cc:118] [619758][TsdProcessOpen]hrtTsdProcessOpen success, subPid[6167], devicePhyId_[2], deviceLogicId_[2], hasBackup[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.311.246 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.311.261 [network_manager.cc:180] [619758][NetworkManager][Init]open tsd by process success, devicePhyId[2], deviceLogicId_[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:50.311.317 [ra_host.c:205]tid:619758,ra_init(205) : Input parameters: phy_id[2], nic_position:[1] hdc_type:[18] [INFO] ROCE(619427,python3.9):2025-04-03-09:35:50.311.416 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] DRV(619427,python3.9):2025-04-03-09:35:50.311.436 [devdrv_hccl.c:103][ascend][curpid:619427,619758][drv][devmng][drvGetProcessSign]Process sign list add success.(tgid=619427) [INFO] ROCE(619427,python3.9):2025-04-03-09:35:50.311.485 [dl_hal_function.c:233]dl_hal_init(233) : dl_hal_init success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:50.311.502 [ra_hdc.c:580]tid:619758,ra_hdc_init(580) : hdc init start! logic id is 2, phy id is 2, hdc_type is 18 [INFO] DRV(619427,python3.9):2025-04-03-09:35:50.311.689 [hdc_core.c:1982][ascend][curpid:619427,619758][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4275; pid=619427) [INFO] HCCP(619427,python3.9):2025-04-03-09:35:50.312.100 [ra_hdc.c:616]tid:619758,ra_hdc_init(616) : hdc init OK! phy_id[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.502 [adapter_hccp.cc:583] [619758]init ra success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.546 [network_manager.cc:222] [619758]NetworkManager nicDeploy[1] deviceLogicId_[2] devicePhyId_[2] init ra OK, nicSocketMap size[0], isHostUseDevNic[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.575 [hccl_communicator_attrs.cc:1307] [619758][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.589 [hccl_communicator_attrs.cc:1275] [619758]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.613 [hccl_communicator.cc:2259] [619758][HcclCommunicator][GetLocalNicPort] nicType[0], devicePortSwitchOn[0], isUseRankPort[1], get port[16666], devId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.671 [adapter_hccp.cc:1650] [619758]hrtRaGetInterfaceVersion phyId[0], opCode[55], version[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.315.923 [adapter_hccp.cc:2343] [619758]Get available Vnic info success, deviceIdType[0] deviceId[0x2] Vnic ip[192.168.2.197] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.008 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:50.316.038 [ra_host.c:280]tid:619758,ra_socket_init(280) : socket init:mode=1 phy_id=2 family=2 ip=192.168.2.197 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.055 [adapter_hccp.cc:806] [619758]socket init success, ip[3305285824] device id[2], socketHandle[0xfffe880ffbb0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.068 [network_manager.cc:692] [619758]devicePhyId[2], ip[192.168.2.197] socket init OK [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.079 [network_manager.cc:432] [619758][Start][Vnic]trying to listen on ip[192.168.2.197] port[16666]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:50.316.095 [ra_host.c:875]tid:619758,ra_socket_listen_start(875) : Input parameters: [0]th, phy_id[2], local_ip[192.168.2.197], port[16666] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.417 [network_manager.cc:448] [619758][Start][Vnic]Listen on ip[192.168.2.197], port[16666] sucess, devPhyId[2], devLogicId[2], isAutoPort[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.438 [network_manager.cc:451] [619758]NetworkManager devicePhyId_[2] port[16666] start vnic OK. refCount[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.451 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.462 [hccl_socket.cc:106] [619758][HcclSocket][Listen] device[2] listens on ip[192.168.2.197] port[16666] success, socketType[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.474 [hccl_socket_manager.cc:56] [619758][Init][Server]ip[192.168.2.197] port[16666] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.495 [hccl_communicator.cc:1548] [619758][HcclCommunicator][InitRaResource] init vnic with ip[192.168.2.197] port[16666] success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.511 [hccl_communicator_attrs.cc:1307] [619758][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.523 [hccl_communicator_attrs.cc:1275] [619758]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.551 [hccl_communicator.cc:1565] [619758]isUsedRdmaLevel0_[0] nicNum[1] hostIP[192.168.0.44], nicDeployment[1]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.564 [network_manager.cc:69] [619758]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.577 [hccl_communicator.cc:2028] [619758]profiling config information:options[], mode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.703 [adapter_trace.cc:21] [619758]Call TraceCreate, return value[0] [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.316.726 [atrace_api.c:61](tid:619758) AtraceCreateWithAttr start. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.316.824 [trace_rb_log.c:45](tid:619758) init ring buffer time finished, realTime 1743644150316816472, monotonicTime 4211952674911, freq 99994 kHz, minutesWest 480 minutes [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.316.840 [trace_rb_log.c:137](tid:619758) [HCCL_619758_2] create ring buffer success, msgSize 128 bytes, msgTxtSize 112 bytes, bufferSize 1024 bytes, msg space 128 bytes, total space 131568 bytes [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.316.857 [tracer_mgr_operate.c:66](tid:619758) create object HCCL_619758_2 success, exitSave(true), noLock(false). [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.316.869 [atrace_api.c:63](tid:619758) AtraceCreateWithAttr end. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.879 [adapter_trace.cc:70] [619758]Call TraceCreateWithAttr, return value[0xaaaad6499b78], Params: objName[HCCL_619758_2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.316.924 [rank_consistentcy_checker.cc:32] [619758][GetInstance] get deviceLogicId[2] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.316.965 [api_impl.cc:1486] 619758 GetMaxStreamAndTask: maxTaskNum:2013, maxStrCount:2016. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.316.989 [npu_driver.cc:4604] 619758 GetMaxStreamAndTask: deviceId=2, tsId=0, streamCount=1984. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.003 [task_exception_handler.cc:1212] [619758]get from RTS the max stream count[1984] the max task count[65535] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.022 [hccl_communicator_attrs.cc:1099] [619758][Set][HcclCommunicatorAttrs][MeshAggregationRankSize]set MeshAggregationRankSize[8]. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.317.077 [api_impl.cc:4219] 619758 RegProfCtrlCallback: Reg profiling callback, moduleId=3. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.098 [command_handle.cc:25] [619758][Profiling][CommandHandle] CommandHandle's rtType is 2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.109 [command_handle.cc:99] [619758][Check][Param]SetMsprofReporterCallback. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.214 [profiler_manager_impl.cc:76] [619758][ProfilerManager][InitProfiler]Register CtrlCallBack success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.231 [hccl_communicator.cc:2193] [619758][BASE][InitProfiler]Register CtrlCallBack success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.317.243 [hccl_communicator.cc:803] [619758]Will use ffts mode. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.317.270 [api_impl.cc:3138] 619758 GetDeviceIndexByPhyId: get Index by PhyId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.317.283 [api_impl.cc:3149] 619758 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.317.293 [api_impl.cc:3157] 619758 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.317.302 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.318.694 [api_impl.cc:3476] 619758 GetDeviceInfo: get device info, deviceId=2 [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:50.318.743 [tbe_vector_reduce.cc:508] [619758]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.319.839 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.007 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.320.026 [dispatcher.cc:157] [619758][GetNotifyMaxWaitTime] notifyMaxWaitTime_ is 2147483647s [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.043 [api_impl.cc:3138] 619758 GetDeviceIndexByPhyId: get Index by PhyId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.054 [api_impl.cc:3149] 619758 GetDeviceIndexByPhyId: GetDeviceIndexByPhyId, device_mode=0, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.064 [api_impl.cc:3157] 619758 GetDeviceIndexByPhyId: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.074 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.100 [api_impl.cc:1990] 619758 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.942 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280000000, end ptr=0x12c280000004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.320.999 [api_impl.cc:1990] 619758 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.321.020 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280001000, end ptr=0x12c280001004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.321.051 [api_impl.cc:1990] 619758 DevMalloc: size=4, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.321.066 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=4, type=16, moduleId=3, start ptr=0x12c280002000, end ptr=0x12c280002004 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.095 [opexecounter.cc:71] [619758]alloc counter mem resource. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.132 [hccl_communicator_attrs.cc:1368] [619758][CollectAlgoAttr]:isUsedRdmaLevel0:[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.180 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.201 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.224 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.238 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.251 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.264 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.278 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.292 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.302 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.315 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.341 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.355 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[3] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.370 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.383 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.395 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[4] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.407 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.421 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.433 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.444 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.455 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.467 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.480 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.493 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.505 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[7] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.517 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.532 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.543 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.555 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.569 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.580 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[9] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.592 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.604 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.618 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[10] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.635 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.649 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.660 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[11] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.674 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.688 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.699 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[12] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.711 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.723 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.736 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[13] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.750 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.765 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.778 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[14] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.790 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.804 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.815 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[15] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.830 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.844 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.856 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[16] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.868 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.881 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.894 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[17] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.907 [alg_configurator.cc:68] [619758][Set][AlgType] isConfigAHC[0] isConfigNULL[0] multiModuleDiffDeviceNumMode[0] multiSuperPodDiffServerNumMode[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.930 [alg_configurator.cc:241] [619758][AlgConfigurator][GetDefaultAlgoLevel1V1] algType[2], moduleNum[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.942 [alg_configurator.cc:134] [619758]Device Type[2], average device count[8], HccsNum[8], SIONum[0], HCCS_SW_NUM[0], optype[18] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.963 [alg_configurator.cc:449] [619758][AlgConfigurator][GetTopoTypeByAlgType]algtype[level0:fullmesh,level1:ring,level2:H-D], devicetype[2],topotype[8] is selected [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.321.977 [alg_configurator.cc:42] [619758]alg configurator init success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.005 [topo_info_extractor.cc:66] [619758]factory init:collective id[192.168.0.44%enp67s0f5_60008_0_1743644144960128], user rank[2], user rank size[8], topo type[8], device Type[2], meshAggregationRankSize[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.028 [topo_info_extractor.cc:155] [619758][SetRankInfo]rankData_: userRank[2], devicePhyId[2], serverIdx[0], superPodId[], superPodIdx[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.051 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=0, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.078 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[0] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.109 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=1, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.128 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[1] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.147 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=3, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.164 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[3] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.197 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=4, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.216 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[4] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.231 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=5, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.246 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[5] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.260 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=6, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.276 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[6] linkTypeRaw[0] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.322.290 [api_error.cc:3478] 619758 GetPairPhyDevicesInfo: input physic devId=2, input physic otherDevId=7, infoType=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.304 [adapter_rts.cc:940] [619758][hrtGetPairDeviceLinkTypeRaw]phyDevId[2] otherPhyDevId[7] linkTypeRaw[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.326 [topo_info_extractor.cc:348] [619758]check server info:server rank size[8], expected value[8], topo type[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.464 [topo_info_extractor.cc:888] [619758]SetTopoInfoForLevel0: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[2], userRankSize[8], topoRankInfo[userRank/devicePhyId: 0/0; 1/1; 2/2; 3/3; 4/4; 5/5; 6/6; 7/7; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.494 [topo_info_extractor.cc:585] [619758][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.524 [topo_info_extractor.cc:646] [619758][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.541 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.558 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.575 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/1; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.589 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.604 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.621 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.634 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.650 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.661 [topo_info_extractor.cc:689] [619758]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[2], userRankSize[8], plane size[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.673 [topo_info_extractor.cc:697] [619758][Set][TopoInfoForLevel2] select origin proc [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.687 [topo_info_extractor.cc:734] [619758]SetTopoInfoForLevel2: topoRankInfo[userRank/devicePhyId/serverIdx/superPodId: 2/2/0/; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.698 [topo_info_extractor.cc:739] [619758]SetTopoInfoForLevel2: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[2], userRankSize[8], plane size[1] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.711 [topo_info_extractor.cc:585] [619758][Set][TopoInfoForLevel1] topoType_[8] ringSize[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.727 [topo_info_extractor.cc:646] [619758][Set][TopoInfoForLevel1] calc subGroup superPodId[] subIndex[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.739 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 0, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 0/192.168.0.44/0/192.168.100.101/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.762 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 1, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 1/192.168.0.44/1/29.105.19.202/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.777 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 2, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 2/192.168.0.44/2/29.105.165.16/1; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.791 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 3, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 3/192.168.0.44/3/29.105.51.138/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.805 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 4, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 4/192.168.0.44/4/29.105.24.184/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.825 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 5, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 5/192.168.0.44/5/29.105.72.114/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.842 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 6, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 6/192.168.0.44/6/29.105.134.38/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.856 [topo_info_extractor.cc:685] [619758]SetTopoInfoForLevel1: topoRankInfo[ ringIndex: 7, userRank/serverId/devicePhyId/nicIp/isBridgeRank: 7/192.168.0.44/7/29.105.73.96/0; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.866 [topo_info_extractor.cc:689] [619758]SetTopoInfoForLevel1: identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], userRank[2], userRankSize[8], plane size[8] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.904 [topo_info_extractor.cc:790] [619758]SetTopoInfoForMeshL1: topoRankInfo[userRank/devicePhyId2/2; ] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.322.920 [topo_info_extractor.cc:519] [619758]plane info:topo type[8], device type[2], COMM_COMBINE size[1], COMM_LEVEL0 size[7], COMM_LEVEL1 size[8], COMM_LEVEL2 size[1], COMM_MESH_L0 size[1], COMM_MESH_L1 size[1] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.323.055 [api_impl.cc:1990] 619758 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.323.082 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280003000, end ptr=0x12c280003020 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.323.130 [hccl_impl.cc:59] [619758]start hccl resources build:get deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.323.263 [hccl_impl.cc:177] [619758]hcclImpl init success. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:50.323.294 [hccl_aiv.cc:279] [619758][AIV][GetAivOpBinaryPath]ENV:LD_LIBRARY_PATH lack fwkacllib/lib64 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.241 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.305 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.411 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.431 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.449 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.471 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.491 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.587 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.603 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.616 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.633 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.652 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.747 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.776 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.793 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.809 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.829 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.919 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.933 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.945 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.960 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.326.980 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.073 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.088 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.100 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.115 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.134 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.226 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.241 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_reduce_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.255 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_reduce_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.269 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.286 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.376 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.391 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.404 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.417 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.435 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.523 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.538 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.560 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.575 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.593 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.682 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.698 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.712 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.726 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.747 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.837 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.852 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.864 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.879 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.901 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.327.993 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.007 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.021 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.035 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.057 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.155 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.185 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.199 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.212 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.231 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.316 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.331 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.345 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.358 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.384 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.473 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.487 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.501 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.515 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.534 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.623 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.638 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_vc_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.651 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_vc_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.665 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.685 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.775 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.788 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.801 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.814 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.833 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.924 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.938 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.951 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.964 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.328.982 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.071 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.085 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.099 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.112 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.130 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.231 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.246 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.259 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.275 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.294 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.384 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.399 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.413 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.429 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.447 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.536 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.550 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.563 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.577 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.595 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.684 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.698 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.712 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.726 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.745 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.832 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.848 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.861 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.876 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.894 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.329.986 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.002 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.023 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.037 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.055 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.144 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.157 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.169 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.184 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.203 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.290 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.304 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.317 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.331 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.348 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.438 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.451 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.463 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.478 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.498 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.592 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.610 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.623 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.639 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.660 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.748 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.762 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.775 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.800 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.819 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.907 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.923 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.937 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.950 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.330.968 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.059 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.074 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.089 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.102 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.123 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.214 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.228 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.243 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.256 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.276 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.363 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.378 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_v_sp_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.392 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_v_sp_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.406 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.424 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.511 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.528 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.541 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.555 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.582 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.676 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.691 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.704 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.718 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.738 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.825 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.843 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.857 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.873 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.894 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.983 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.331.999 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.011 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.025 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.044 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.131 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.147 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.169 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.189 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.210 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.302 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.318 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.333 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.346 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.365 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.456 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.481 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.497 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.512 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.530 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.617 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.632 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.646 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.661 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.680 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.771 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.787 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_to_all_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.802 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_to_all_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.818 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.836 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.928 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.944 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.961 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.980 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.332.997 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.090 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.105 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.118 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.131 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.149 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.239 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.255 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.277 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.291 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.312 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.403 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.418 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.432 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.446 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.464 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.553 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.569 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.582 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.597 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.618 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.707 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.723 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.736 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.750 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.770 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.859 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.874 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.888 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.901 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.333.920 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.009 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.023 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.036 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.057 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.077 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.171 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.186 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.199 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.215 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.233 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.323 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.339 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.353 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.368 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.387 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.478 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.494 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.508 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.522 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.540 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.628 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.642 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_reduce_scatter_v_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.654 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_reduce_scatter_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.668 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.688 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.778 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.797 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.810 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.826 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.844 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.940 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.956 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.971 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.334.985 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.003 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.092 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.109 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.123 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.138 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.156 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.245 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.262 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.276 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.290 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.308 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.399 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.416 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.429 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.446 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.466 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.557 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.574 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.588 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.603 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.623 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.715 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.739 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.752 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.766 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.784 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.873 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.889 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.903 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.917 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.335.937 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.027 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.043 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.058 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.074 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.096 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.201 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.219 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_half. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.234 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_half, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.250 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.269 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.360 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.375 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.388 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_int16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.403 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.423 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.512 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.526 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.538 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_uint16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.559 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.578 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.671 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.687 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_float. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.703 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_float, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.718 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.736 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.825 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.841 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.853 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_int32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.868 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.887 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.974 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.336.990 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint32_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.003 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_uint32_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.018 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.038 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.126 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.143 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_int8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.157 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_int8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.171 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.190 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.278 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.295 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_uint8_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.309 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_uint8_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.323 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.350 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.440 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.455 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=aiv_all_gather_v_bfloat16_t. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.471 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=aiv_all_gather_v_bfloat16_t, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.485 [runtime.cc:1673] 619758 MallocProgramAndReg: MAGIC_ELF_AIVEC new ElfProgram ok, Runtime_alloc_size 288 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.503 [elf.cc:566] 619758 ParaseKernelMetaData: dataFlag=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.592 [runtime.cc:1688] 619758 MallocProgramAndReg: isSupportMix=0, kernelType=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.606 [runtime.cc:1953] 619758 KernelRegister: Runtime_alloc_size 232, type=2, funcMode=0, kernelInfoExt=hccl_aiv_sync. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.619 [runtime.cc:1897] 619758 KernelSetDfx: kernel_name=hccl_aiv_sync, dfxAddr=0, dfxSize=0, elfDataFlag=0 [INFO] DRV(619427,python3.9):2025-04-03-09:35:50.337.650 [devmm_svm.c:1972][ascend][curpid:619427,619758][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.337.666 [hdc.cc:83] [619758][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.679 [api_impl.cc:1990] 619758 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.337.703 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c280004000, end ptr=0x12c2800a5000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.224 [api_impl.cc:1990] 619758 DevMalloc: size=659456, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.243 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=659456, type=16, moduleId=3, start ptr=0x12c2800a5000, end ptr=0x12c280146000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.261 [api_impl.cc:2664] 619758 MemSetSync: destMax=659456, value=0, count=659456 [INFO] DRV(619427,python3.9):2025-04-03-09:35:50.338.423 [devmm_svm.c:1972][ascend][curpid:619427,619758][drv][devmm][devmm_ctrl_support_feature]RTS note drv support feature. (in_feature=0x2; out_feature=0x2) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.338.441 [hdc.cc:83] [619758][HDCommunicate]supportDevMemReg_ 1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.453 [api_impl.cc:1990] 619758 DevMalloc: size=4096, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.467 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=4096, type=16, moduleId=3, start ptr=0x12c280146000, end ptr=0x12c280147000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.488 [api_impl.cc:2096] 619758 HostMalloc: size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.504 [api_error.cc:1260] 619758 HostMalloc: Host memory malloc succeed,size=4096 , moduleId=3, host addr=0x12c0807f0000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.703 [api_impl.cc:1990] 619758 DevMalloc: size=2100520, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.776 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=2100520, type=16, moduleId=3, start ptr=0x12c100c00000, end ptr=0x12c100e00d28 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.795 [api_impl.cc:2664] 619758 MemSetSync: destMax=2100520, value=0, count=2100520 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.931 [api_impl.cc:1990] 619758 DevMalloc: size=32, type=16, moduleId=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.960 [api_error.cc:1218] 619758 DevMalloc: device malloc Succ, size=32, type=16, moduleId=3, start ptr=0x12c280147000, end ptr=0x12c280147020 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.338.975 [api_impl.cc:2664] 619758 MemSetSync: destMax=32, value=0, count=32 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.003 [peterson_lock.cc:69] [619758][PetersonLock][AllocDeviceMem] Type[Host] alloc memSize[32 Byte] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.017 [peterson_lock.cc:47] [619758][PetersonLock][Init] type [Host] init success, memSize [32 Byte] timeout[60 s] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.049 [hccl_comm.cc:129] [619758]hcclCommInitInfo:commId[hccl-619427-0-1743644148-hccl_world_group], rank[2], totalRanks[8], serverId[192.168.0.44%enp67s0f5], deviceType[2],logicDevId[2], identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.068 [hccl_comm.cc:175] [619758][SetQpQosAttr]The TC and SL do not use the config configuration. It will use environment variables to configure. TC[132], SL[4] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.084 [topo_matcher.cc:528] [619758][SetAivMode]AivMode is set to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.097 [topo_matcher.cc:540] [619758][SetAicpuMode]Aicpu is set to [0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.114 [hcom_plugin.cc:349] [619758][Set][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] rankSize[8]. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.339.142 [runtime.cc:5239] 619758 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.154 [adapter_rts.cc:198] [619758][hrtGetDeviceRefresh]deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.165 [hcom_common.cc:85] [619758][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.191 [op_base.cc:128] [619758][HcclGetDeviceId] deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.205 [hccl_comm.cc:1296] [619758]HCCL_KEY_INFO: GetTopoDesc topoDescs[0xfffea97accb8] topoSize[2] [INFO] GE(619427,python3.9):2025-04-03-09:35:50.339.227 [hcom_topo_info.cc:24]619758 SetGroupTopoInfo:Add group 192.168.0.44%enp67s0f5_60008_0_1743644144960128 successfully. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.239 [hcom_plugin.cc:371] [619758][Set][GroupTopoInfo]SetGroupTopoInfo group set success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.263 [topoinfo_detect.cc:555] [619758]NetworkManager stop host net success! ip[192.168.0.44%enp67s0f5] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.274 [topoinfo_detect.cc:321] [619758]TopoInfoDetect TeardownAgent ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.286 [topoinfo_detect.cc:219] [619758]TopoInfoDetect TeardownServer ok, identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.308 [op_base.cc:1050] [619758][InitCommRootInfo]Init success, rankNum[8], rank[2], rootInfo identifier[192.168.0.44%enp67s0f5_60008_0_1743644144960128], server[192.168.0.44%enp67s0f5], logicDevId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.324 [profiler_base.cc:342] [619758]AddGroupUdi: group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] udi[] deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.338 [op_base.cc:1107] [619758][HCCL_TRACE]HcclCommInitRootInfoInner success, take time [4978186]us, rankNum[8], rank[2] [WARNING] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.339.402 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:157] InitializeByRootInfoConfig] End to initialize communicator by HcclCommInitRootInfo for hccl_world_group [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.339.430 [op_base.cc:853] [619758]HcclGetCommName input handle=0xfffe88022f40 commName=192.168.0.44%enp67s0f5_60008_0_1743644144960128 [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.339.470 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:93] Initialize] Start initializing hccl watchdog on device side for group: hccl_world_group, rank: 2 [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.339.494 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:77] Initialize] Initialize hccl watch dog handler. rank id: 2, group name: hccl_world_group [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.339.622 [mindspore/ccsrc/plugin/res_manager/ascend/collective/ascend_communication_group.cc:98] Initialize] hccl watchdog on device side is successfully initialized. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.339.642 [device.cpp:183]619758 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 2. [INFO] DEVICE(619427,fffe8fffefa0,python3.9):2025-04-03-09:35:50.339.663 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:155] WatchDogProcess] WatchDogProcess start, rank id: 2, group name: hccl_world_group [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.339.678 [api_impl.cc:3419] 619758 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.339.691 [runtime.cc:3444] 619758 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.339.704 [api_impl.cc:3449] 619758 DeviceReset: Succ, userDeviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.339.714 [device.cpp:189]619758 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 2. [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.339.734 [mindspore/ccsrc/plugin/res_manager/ascend/collective/multi_ascend_communication_group.cc:58] Initialize] Successfully initialize HCCL group hccl_world_group [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:50.339.797 [mindspore/ccsrc/distributed/collective/collective_manager.cc:824] CreateDeviceCommunicator] [PROF]InitDeviceCommunicator costs 5188.52 msec. [WARNING] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:50.339.825 [mindspore/ccsrc/distributed/collective/collective_manager.cc:828] CreateDeviceCommunicator] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.339.879 [mindspore/ccsrc/distributed/collective/collective_manager.cc:404] CreateCommunicationGroup] [PROF]distributed_create_group costs 5690.09 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.339.907 [mindspore/ccsrc/distributed/collective/collective_manager.cc:225] Initialize] [PROF]CreateGlobalCommunicationGroup costs 5690.13 msec. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.339.925 [mindspore/ccsrc/distributed/collective/collective_manager.cc:228] Initialize] End initializing collective communication for backend: Ascend [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.339.950 [mindspore/ccsrc/distributed/init.cc:62] Initialize] [PROF]distributed_collective_init costs 9837.13 msec. [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.339.979 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:254] RecordInitStatus] Status record: system init. =================================================================================================================== test session starts ==================================================================================================================== platform linux -- Python 3.9.19, pytest-6.2.5, py-1.11.0, pluggy-1.5.0 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/lccl plugins: requests-mock-1.12.1, hydra-core-1.3.2, mock-3.14.0, forked-1.6.0, xdist-1.32.0 collected 1 item test_lccl_allreduce.py [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.355.985 [device.cpp:143]619427 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.039 [api_impl.cc:2999] 619427 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.054 [device.cc:306] 619427 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.069 [runtime.cc:3393] 619427 PrimaryContextRetain: Context inc ref, ts_id=0, count=0x9 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.084 [runtime.cc:3560] 619427 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.097 [api_impl.cc:2987] 619427 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.110 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.128 [api_impl.cc:2992] 619427 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.148 [api_impl.cc:3025] 619427 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.177 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.192 [acl.cpp:121]619427 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.207 [device.cpp:154]619427 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.356.251 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.272 [context.cpp:174]619427 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.292 [api_impl.cc:6913] 619427 CtxSetSysParamOpt: curCtx = 0xaaaae7011ab0, configOpt=0, configVal=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.306 [context.cpp:158]619427 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.356.920 [op_base.cc:1214] [619427][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.356.943 [op_base.cc:128] [619427][HcclGetDeviceId] deviceLogicId[2] [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.356.967 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.356.984 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.357.006 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.357.046 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.357.623 [mindspore/ccsrc/runtime/pynative/op_compiler.cc:179] Compile] Run Op cache miss Ascend_0_AllReducehccl_world_group0sumtrue03-1-3-343_0 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.148 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.222 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.390 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 177.141 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.433 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.484 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 47.42 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.503 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.520 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 16.42 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.536 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 49.69 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.624 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.479 msec. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.728 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:191] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.787 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.32 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.863 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 49 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.905 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:204] EliminateIllegalDataTypePass] [PROF]EliminateIllegalDataTypePass costs 0.174 msec. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.934 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:151] CommonUnifyMindIR] start common unify mindir opt graph:4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.358.972 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.025 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 49.73 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.045 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.076 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 28.73 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.096 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.113 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 16.13 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.130 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.162 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 29.69 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.215 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:164] CommonUnifyMindIR] [PROF]CommonUnifyMindIR costs 0.277 msec. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.247 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:81] BackendCommonOptimization] Status record: start common optimization. graph id: 4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.352 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.432 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_0_convert_dynamic_broadcast_to in 77.57 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.492 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_1_convert_const_input_to_attr in 34.57 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.536 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_2_custom_op_const_input_to_attr in 20.93 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_3_convert_const_input_to_tensor_input in 28.74 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.649 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 37.531 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.784 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 111.39 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.810 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.99 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_7_flatten_concat_fission is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.914 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_7_flatten_concat_fission in 66.54 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.966 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 26.82 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.359.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_9_broadcast_to_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.055 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_9_broadcast_to_fusion in 46.2 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.533 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 450.201 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.564 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_common_pm_11_replace_addn is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.622 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_common_pm_11_replace_addn in 39.03 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.673 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:93] BackendCommonOptimization] [PROF]BackendCommonOptimization costs 1.42 msec. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.701 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:100] BackendCommonOptimization] Status record: end common optimization. graph id: 4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.360.755 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:258] OptimizationWithoutBackend] [PROF]OptimizationWithoutBackend costs 2.026 msec. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.186 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_0_renorm_split in 57.79 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.214 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.297 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_1_reduce_axis_update in 79.91 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.319 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.378 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_2_space_to_batch_nd_attr_update in 55.79 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.399 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_3_batch_to_space_nd_attr_update in 31.84 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.476 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AdamWeightDecay not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.497 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_4_adam_weight_decay_unify_mindir in 40.47 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.540 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_5_add_depend_for_adamw in 24.46 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.596 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.616 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_6_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 54.57 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.633 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.650 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_7_pynative_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 16.25 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.667 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SparseSoftmaxCrossEntropyWithLogits not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.684 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_8_pynative_sparse_softmax_cross_entropy_with_logits_unify_mindir in 17.01 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.715 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutExt not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.744 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_9_dropout_ext_unify_mindir1 in 43.17 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.777 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim DropoutGradExt not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.793 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_10_dropoutgrad_ext_unify_mindir in 32.31 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.809 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Dropout not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.825 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_11_dropout_unify_mindir1 in 14.54 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.841 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.879 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_12_dropoutgrad_unify_mindir in 34.51 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.914 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_13_neighbor_exchange_unify_mindir in 34.31 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.949 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.965 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_14_neighbor_exchange_v2_unify_mindir in 16.09 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.983 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.361.997 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_15_neighbor_exchange_v2_grad_unify_mindir in 12.67 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.012 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.027 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_16_all_to_all_unify_mindir in 14.26 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.063 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AlltoAllV not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.081 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_17_all_to_all_v_unify_mindir in 33.84 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.115 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.132 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_18_bn_split in 33.9 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.146 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:389] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.209 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_19_bn_grad_unify_mindir in 58.81 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.253 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.274 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_20_bn_grad_split in 33.67 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.294 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.309 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_21_batchnormgrad_to_bninfergrad in 14.87 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.324 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.338 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_22_batch_norm_grad_infer_fission in 13.01 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.352 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.367 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_23_batchnorm_to_bninfer in 13.82 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.383 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim Print not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.399 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_24_print_insert_placeholder_for_tensor_name in 15.87 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.432 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim GetNext not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.448 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_25_getnext_for_ge in 31.63 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.479 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNorm not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.494 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_26_sync_bn_split in 28.65 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.511 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim SyncBatchNormGrad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.526 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_27_sync_bn_grad_split in 14.64 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.541 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:398] Run] Prim AvgPoolGrad not exist in name to cnode [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.557 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_28_avg_pool_grad_for_ge in 14.88 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.650 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_29_add_attr_to_dump in 71.42 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.891 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unify_mindir_30_ascend_mindir_op_adapter in 212.81 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.915 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.362.980 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_0_clip_by_norm_fission in 35.56 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.002 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_1_cdist_fission is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.073 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_1_cdist_fission in 51.091 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.094 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_2_cdist_grad_fission is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.139 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_2_cdist_grad_fission in 26.91 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.161 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.211 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_3_batchmatmul_reducescatter_alltoall_fusion in 25.55 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.227 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.267 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_4_alltoall_allgather_batch_matmul_fusion in 23.83 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.283 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_5_lamb_fission_ge is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.367 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_5_lamb_fission_ge in 67.55 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.389 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_6_adaptive_max_pool2d_ge_fusion in 47.55 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.480 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.576 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_7_matmul_reduce_scatter_fusion in 77.4 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.597 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.669 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_8_allgather_matmul_fusion in 55.14 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.693 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.725 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:270] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.853 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_9_FlashAttentionFusionV1 in 127.291 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.874 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.363.892 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:384] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.017 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_10_FlashAttentionFusionV2 in 122.97 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.039 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.131 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_11_quantbatchmatmul_reduce_fusion in 72.32 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.154 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_12_MatMulAllReduce is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.255 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_12_MatMulAllReduce in 60.93 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.278 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_13_grouped_matmul_assignadd_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.295 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_14_MatMulAllReduceAddRmsNorm is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.310 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_15_add_layer_norm_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.323 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_16_add_layer_norm_v3_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.339 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_17_add_layer_norm_ext_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.352 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_18_inference_qbmm_add_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.369 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_19_inference_swiglu_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.384 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_20_inference_matmul_split_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.396 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_21_add_rms_norm_dynamic_quant_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.420 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_22_add_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_23_add_cast_rms_norm_cast_quant_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.448 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_24_rms_norm_quant_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.462 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_25_add_cast_rms_norm_cast_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.477 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_26_shape_reshape is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.490 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_27_split_concat_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.750 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_27_split_concat_fusion in 240.04 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.774 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_28_matmul_sigmoid_add_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.790 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_29_matmul_sigmoid_cast_add_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.805 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_30_matmul_elemwise_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.820 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_31_qbmm_allreduce_convert_bias is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.834 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_32_inference_qbmm_allreduce_add_fusion in 78.68 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.952 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_33_remove_fa_tensor_to_tuple_ops is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.971 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_backend_fusion_34_transpose_batch_matmul_transpose_fusion is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.364.985 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.365.064 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_backend_fusion_35_add_rms_norm_fusion in 59.94 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.365.119 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:203] GEUnifyMindIR] [PROF]GEUnifyMindIR costs 4.314 msec. [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.365.463 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.365.605 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:40] MarkRefGraph] Mark graph is ref graph: 4000000000 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.365.902 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 232.291 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.034 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_0_process_call_inline in 34.28 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.147 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_1_seed_adapter in 82.15 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.185 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_2_insert_tensor_move_for_communication in 12.93 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.241 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_3_process partial inline in 29.28 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.347 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_4_expander_fallback in 80.01 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.403 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_5_convert_pad_v3_paddings in 30.42 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.453 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_6_convert_pad_v3_grad_paddings in 25.45 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.521 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_7_resize_bilinear_add_attr in 45.03 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.551 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_pm_8_backend_custom_depend in 8.52 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.597 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:120] GEBackendOptimizeACL] [PROF]GEBackendOptimizeACL costs 0.624 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.654 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:131] OptimizeACLGraph] [PROF]OptimizeACLGraph costs 1.048 msec. [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.366.699 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.051 [mindspore/ccsrc/plugin/device/ascend/hal/device/kernel_select_ascend.cc:624] CollectOpSelectedType] AllReduce select hccl kernel [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.086 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:315] SelectKernel] [PROF]SelectKernel costs 0.403 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.104 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:319] PrintOpSelectedNum] Number of GE_KERNEL, INTERNAL_KERNEL, OPAPI_KERNEL, ACL_KERNEL, ATB_KERNEL, HCCL_KERNEL, HOST_KERNEL: [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.119 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:327] PrintOpSelectedNum] 0 0 0 0 0 1 0 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.236 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 40.51 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.329 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 65.87 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.400 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 44.58 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.493 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 68.75 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.514 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:119] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_4_shape_reshape is disabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.589 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_opt_acl_after_kernel_select_pm_5_shape_reshape_direct in 38.75 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.623 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_6_graph_view_replace in 13.06 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.644 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_7_label_1f1b_overlap_node in 1.28 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.688 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:173] GEBackendOptimizeACLAfterKernelSelect] [PROF]GEBackendOptimizeACLAfterKernelSelect costs 0.53 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.742 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_optimization.cc:166] OptimizeACLGraphAfterKernelSelect] [PROF]OptimizeACLGraphAfterKernelSelect costs 0.585 msec. [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.785 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.895 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:117] Run] graph kernel pass hwopt_fusion_after_inline_pm_0_DropoutGenMask is enabled. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.921 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_fusion_after_inline_pm_0_DropoutGenMask in 1.3 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.968 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_1_cse in 27.13 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.367.994 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_maketuple_getitem in 3.96 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.015 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_3_insert_move_to in 0.78 us [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.045 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_4_insert_pre_fetch_depend in 85.47 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.154 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:75] RunPass] Run pass hwopt_after_inline_pm_5_overlap_1b1f in 0.63 us [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.202 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:242] GEAfterInlineOptimize] [PROF]GEAfterInlineOptimize costs 0.335 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.247 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:437] InlineCallGraph] [PROF]InlineCallGraph costs 0.479 msec. [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.280 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.357 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:912] InlineSwitchGraph] [PROF]InlineSwitchGraph costs 0.086 msec. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.368.820 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.368.852 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.368.866 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] KERNEL(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.368.906 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:32] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.149 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel.cc:211] CalLoopSize] Get Hccl Kernel: AllReduce, output size: 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.208 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:198] GenerateKernelMod] kernel opname:AllReduce, kernel type:hccl_kernel [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.302 [mindspore/ccsrc/backend/common/session/exec_order_builder.cc:84] Build] exec order build by bfs [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.406 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1179] OptimizeExecutionOrder] [PROF]OptimizeExecutionOrder costs 0.115 msec. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.497 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1214] ResetNodeIds] Start reset node id [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.552 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1234] ResetNodeIds] End reset node id [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.577 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1123] DoStreamAssign] Status record: start stream assign, pynative_kernel_graph4000000000 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.595 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:151] AssignStream] Not stream assign when pynative forward. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.611 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1139] DoStreamAssign] Status record: end stream assign, pynative_kernel_graph4000000000 [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.632 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:944] PrintGraphExecuteOrder] Graph 4000000000 execution order: [INFO] SESSION(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.369.684 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:984] PrintGraphExecuteOrder] Index[0], node name[Default/AllReduce-op0], logic id[4294967295], stream id[0], node info[@pynative_kernel_graph4000000000:CNode_1{[0]: ValueNode AllReduce, [1]: param_Parameter_2}], group[hccl_world_group] [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.370.021 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.370.321 [context.cpp:72]619792 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.370.351 [api_impl.cc:3717] 619792 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.370.370 [awatchdog_core.c:175](tid:619792) create watchdog for id : 65543, tid : 619792, timeout : 300s [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.370.382 [context.cpp:80]619792 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.370.421 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.370.453 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.370.474 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61022928896, align_size:61022928896 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.371.769 [api_impl.cc:7115] 619792 ReserveMemAddress: device malloc Succ, size=61022928896(bytes) start ptr=0x12c2c0000000, end ptr=0x12d0f5400000 [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.378.421 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12c2c0000000,"size_":61022928896,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.460 [api_impl.cc:7129] 619792 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.585 [api_impl.cc:7145] 619792 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12c2c0000000, end ptr=0x12c2c0200000 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.650 [device.cpp:143]619792 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.671 [api_impl.cc:2999] 619792 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.684 [device.cc:306] 619792 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.697 [runtime.cc:3393] 619792 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xa [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.709 [runtime.cc:3560] 619792 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.721 [api_impl.cc:2987] 619792 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.732 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.747 [api_impl.cc:2992] 619792 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.762 [api_impl.cc:3025] 619792 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.775 [acl.cpp:115]619792 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.786 [acl.cpp:121]619792 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.799 [device.cpp:154]619792 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.378.825 [mindspore/ccsrc/plugin/res_manager/ascend/ascend_res_manager.cc:496] SetDeterministic] Set kernel deterministic value: 0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.852 [context.cpp:174]619792 aclrtCtxSetSysParamOpt: start to execute aclrtCtxSetSysParamOpt, opt = 0, value = 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.869 [api_impl.cc:6913] 619792 CtxSetSysParamOpt: curCtx = 0xaaaae7011ab0, configOpt=0, configVal=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.882 [context.cpp:158]619792 SetSysParamOpt: successfully execute aclrtCtxSetSysParamOpt [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.378.926 [op_base.cc:1214] [619792][HcclSetConfig] Set HCCL_DETERMINISTIC is false [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.378.943 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.378.956 [adapter_rts.cc:234] [619792][hrtGetDevice]deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:50.378.966 [op_base.cc:128] [619792][HcclGetDeviceId] deviceLogicId[2] [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.378.997 [context.cpp:72]619792 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.011 [api_impl.cc:3717] 619792 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.379.022 [context.cpp:80]619792 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.050 [api_error.cc:1807] 619792 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.070 [memory_task.cc:342] 619792 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.083 [memory_task.cc:473] 619792 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.094 [memory_task.cc:518] 619792 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.114 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.135 [memory_task.cc:1007] 619792 ConstructPcieDmaSqe: stream_id=2, task_id=0, copyType=0, src=0xaaaaf2c1e040, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.151 [memory_task.cc:1166] 619792 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=0, copyType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.165 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.179 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=0, type=5(MEMCPY_ASYNC), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.201 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=0, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.217 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.379.238 [callback.cpp:97]619792 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.267 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=1, taskResHead_=0, taskResTail_=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.287 [task_info.cc:521] 619792 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=1,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.309 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.324 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=1, type=72(HOSTFUNC_CALLBACK), head=0, tail=2, sqe_num=1, lastTail=1, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.339 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=1, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.352 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.379.362 [callback.cpp:109]619792 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.394 [npu_driver.cc:3844] 619588 CqReportGet: cq report get start, device_id=2, tsId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.420 [npu_driver.cc:3851] 619588 CqReportGet: halCqReportGet count=1 [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.379.428 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:704] GenerateAllocator] Generate allocator, is persistent : 0, stream id : 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.430 [api_impl.cc:5087] 619588 ProcessReport: report[0], sqid=12291, streamId=2, taskId=1, eventId=65535, isBlock=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.379.453 [callback.cpp:138]619588 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.379.456 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:716] operator()] Malloc mem block, is enable eager free : 0, is enable vmm : 1, size : 512, block size : 1073741824. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.379.466 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.379.481 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_vmm_adapter.cc:204] AllocDeviceMem] VMM AllocDeviceMem size:61022928896, align_size:61022928896 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.379.478 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.380.766 [api_impl.cc:7115] 619792 ReserveMemAddress: device malloc Succ, size=61022928896(bytes) start ptr=0x12d100000000, end ptr=0x12df35400000 [INFO] PRE_ACT(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.387.726 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:736] operator()] Malloc mem block : {"addr_":0x12d100000000,"size_":61022928896,"stream_id_":0,"min_addr_":0,"max_addr_":0}. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.387.769 [api_impl.cc:7129] 619792 MallocPhysical: Start to MallocPhysical [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.387.882 [api_impl.cc:7145] 619792 MapMem: device malloc Succ, size=2097152(bytes), start ptr=0x12d100000000, end ptr=0x12d100200000 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.388.016 [device.cpp:231]619792 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.039 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.388.057 [acl.cpp:802]619792 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.187 [npu_driver.cc:404] 619792 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.226 [stream.cc:2246] 619792 AllocExecutedTimesSvm: stream_id=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.263 [stream_sqcq_manage.cc:147] 619792 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.291 [npu_driver.cc:882] 619792 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.583 [npu_driver.cc:6125] 619792 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.636 [npu_driver.cc:6137] 619792 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1010, streamId=6, logicCq=1010, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.663 [npu_driver.cc:6189] 619792 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=6, logicCq=1010. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.683 [npu_driver.cc:5916] 619792 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0144000, addr=dfffe0144000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.697 [stream.cc:653] 619792 Setup: Success to get sq=6 sq reg virtual addr length=4096. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.744 [stream.cc:276] 619792 CreateStreamTaskRes: Normal STREAM Setup stream_id=6 flags_=0x0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.760 [stream.cc:688] 619792 Setup: stream setup end, stream_id=6, IsTaskSink=0, sqId=6, cqId=6, deviceId=2, streamResId=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.388.775 [acl_resource_manager.cpp:814]619792 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.388.788 [acl_resource_manager.cpp:816]619792 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.947 [engine.cc:188] 619792 SubmitTaskNormal: stream_id=6,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.969 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.986 [task_info.cc:409] 619792 ConstructSqeForGetDevMsgTask: stream_id:6, task_id:0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.388.998 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.010 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=6, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.027 [npu_driver.cc:751] 619792 SqTaskSend: device_id=2, ts_id=0, sq_id=6, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.038 [stars_engine.cc:893] 619792 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=6, cq_id=6, stream_id=6, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.052 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.067 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=6, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.080 [stream.cc:2886] 619792 StarsWaitForTask: No.0 stream_id=6, taskId=0, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.093 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=6, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.104 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.118 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.130 [logger.cc:152] 619792 TaskFinished: device_id=2, stream_id=6, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.151 [stars_engine.cc:437] 619792 FinishedTaskReclaim: recycle task, stream_id=6, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.173 [stream.cc:2886] 619792 StarsWaitForTask: No.1 stream_id=6, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.188 [stream.cc:852] 619792 TearDown: stream_id=6, sqId=6, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.203 [stream.cc:938] 619792 TearDown: stream_id=6, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.214 [stream.cc:1507] 619792 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.240 [stream_sqcq_manage.cc:451] 619792 FreeLogicCq: Return(0), threadIdentifier(281469230182304), devId(2), tsId(0), cqId(1010), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.261 [npu_driver.cc:6163] 619792 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=6, logicCq=1010. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.292 [npu_driver.cc:1180] 619792 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1010, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.308 [stream.cc:237] 619792 FreeStreamId: Free stream_id=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.323 [stream_sqcq_manage.cc:216] 619792 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.389.334 [npu_driver.cc:966] 619792 NormalSqCqFree: Normal sqcq free, sqId=6, cqId=6, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.058 [npu_driver.cc:974] 619792 NormalSqCqFree: Normal sqcq free success, sqId=6, cqId=6, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.078 [stream_sqcq_manage.cc:228] 619792 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=6, cq_id=6, stream_id=6, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.094 [npu_driver.cc:432] 619792 StreamIdFree: stream id free begin, device_id=2, stream_id=6, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.110 [npu_driver.cc:441] 619792 StreamIdFree: Stream id free success, stream_id=6, device_id=2, tsId=0, streamFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.128 [device_msg_handler.cc:110] 619792 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.840 [device_msg_handler.cc:127] 619792 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.896 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.390.928 [npu_driver.cc:500] 619792 GetFaultEvent: get fault event, device_id=2, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.391.012 [acl.cpp:782]619792 GetFaultEventInfo: can not get fault event of device 2, runtime errorCode is 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:50.391.039 [error_manager.cc:989]619792 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.391.376 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.391.415 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.391.437 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.391.452 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.391.749 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.391.866 [context.cpp:72]619792 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.888 [api_impl.cc:3717] 619792 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.391.903 [context.cpp:80]619792 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.923 [api_error.cc:1807] 619792 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.939 [memory_task.cc:342] 619792 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.391.943 [mindspore/ccsrc/common/amp/amp.cc:149] GetCurrentAmpStrategy] amp strategy stack is empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.952 [memory_task.cc:473] 619792 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.965 [memory_task.cc:518] 619792 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.977 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=2, taskResHead_=0, taskResTail_=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.391.992 [memory_task.cc:1007] 619792 ConstructPcieDmaSqe: stream_id=2, task_id=2, copyType=0, src=0xaaaaf2c1ea60, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.005 [memory_task.cc:1166] 619792 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=2, copyType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.019 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), head=0, tail=3 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.033 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=2, type=5(MEMCPY_ASYNC), head=0, tail=3, sqe_num=1, lastTail=2, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.049 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=2, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.062 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.081 [callback.cpp:97]619792 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.100 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=3, taskResHead_=0, taskResTail_=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.116 [task_info.cc:521] 619792 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=3,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.128 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.141 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=3, type=72(HOSTFUNC_CALLBACK), head=0, tail=4, sqe_num=1, lastTail=3, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.172 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=3, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.186 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.210 [callback.cpp:109]619792 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.211 [npu_driver.cc:3844] 619588 CqReportGet: cq report get start, device_id=2, tsId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.227 [npu_driver.cc:3851] 619588 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.238 [api_impl.cc:5087] 619588 ProcessReport: report[0], sqid=12291, streamId=2, taskId=3, eventId=65535, isBlock=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.253 [callback.cpp:138]619588 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.265 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.276 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.284 [device.cpp:231]619792 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.300 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.311 [acl.cpp:802]619792 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.340 [npu_driver.cc:404] 619792 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.366 [stream.cc:2246] 619792 AllocExecutedTimesSvm: stream_id=7. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.401 [stream_sqcq_manage.cc:147] 619792 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.417 [npu_driver.cc:882] 619792 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.597 [npu_driver.cc:6125] 619792 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.636 [npu_driver.cc:6137] 619792 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1011, streamId=7, logicCq=1011, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.656 [npu_driver.cc:6189] 619792 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=7, logicCq=1011. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.670 [npu_driver.cc:5916] 619792 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0144000, addr=dfffe0144000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.682 [stream.cc:653] 619792 Setup: Success to get sq=7 sq reg virtual addr length=4096. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.716 [stream.cc:276] 619792 CreateStreamTaskRes: Normal STREAM Setup stream_id=7 flags_=0x0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.730 [stream.cc:688] 619792 Setup: stream setup end, stream_id=7, IsTaskSink=0, sqId=7, cqId=7, deviceId=2, streamResId=65534 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.742 [acl_resource_manager.cpp:814]619792 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.392.751 [acl_resource_manager.cpp:816]619792 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.788 [engine.cc:188] 619792 SubmitTaskNormal: stream_id=7,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.805 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.820 [task_info.cc:409] 619792 ConstructSqeForGetDevMsgTask: stream_id:7, task_id:0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.849 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.864 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=7, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.880 [npu_driver.cc:751] 619792 SqTaskSend: device_id=2, ts_id=0, sq_id=7, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.892 [stars_engine.cc:893] 619792 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=7, cq_id=7, stream_id=7, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.904 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.915 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=7, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.928 [stream.cc:2886] 619792 StarsWaitForTask: No.0 stream_id=7, taskId=0, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.938 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=7, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.951 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.963 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.974 [logger.cc:152] 619792 TaskFinished: device_id=2, stream_id=7, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=7 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.392.987 [stars_engine.cc:437] 619792 FinishedTaskReclaim: recycle task, stream_id=7, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.000 [stream.cc:2886] 619792 StarsWaitForTask: No.1 stream_id=7, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.010 [stream.cc:852] 619792 TearDown: stream_id=7, sqId=7, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.023 [stream.cc:938] 619792 TearDown: stream_id=7, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.033 [stream.cc:1507] 619792 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.053 [stream_sqcq_manage.cc:451] 619792 FreeLogicCq: Return(0), threadIdentifier(281469230182304), devId(2), tsId(0), cqId(1011), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.070 [npu_driver.cc:6163] 619792 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=7, logicCq=1011. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.097 [npu_driver.cc:1180] 619792 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1011, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.111 [stream.cc:237] 619792 FreeStreamId: Free stream_id=7. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.123 [stream_sqcq_manage.cc:216] 619792 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.133 [npu_driver.cc:966] 619792 NormalSqCqFree: Normal sqcq free, sqId=7, cqId=7, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.574 [npu_driver.cc:974] 619792 NormalSqCqFree: Normal sqcq free success, sqId=7, cqId=7, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.591 [stream_sqcq_manage.cc:228] 619792 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=7, cq_id=7, stream_id=7, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.603 [npu_driver.cc:432] 619792 StreamIdFree: stream id free begin, device_id=2, stream_id=7, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.631 [npu_driver.cc:441] 619792 StreamIdFree: Stream id free success, stream_id=7, device_id=2, tsId=0, streamFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.645 [device_msg_handler.cc:110] 619792 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.934 [device_msg_handler.cc:127] 619792 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.965 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.393.985 [npu_driver.cc:500] 619792 GetFaultEvent: get fault event, device_id=2, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.042 [acl.cpp:782]619792 GetFaultEventInfo: can not get fault event of device 2, runtime errorCode is 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:50.394.059 [error_manager.cc:989]619792 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.394.348 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.394.379 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.394.396 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.394.409 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.394.476 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.550 [context.cpp:72]619792 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.572 [api_impl.cc:3717] 619792 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.587 [context.cpp:80]619792 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.603 [api_error.cc:1807] 619792 MemcpyAsyncCheckKindAndLocation: src pointer loc type=1, dst pointer loc type=2, kind=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.619 [memory_task.cc:342] 619792 ConvertCpyType: MemcpyAsyncTask::ConvertCpyType MEMCPY_HOST_TO_DEVICE, direct=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.630 [memory_task.cc:473] 619792 MemcpyAsyncTaskInitV3: Init qos=0, partId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.641 [memory_task.cc:518] 619792 MemcpyAsyncTaskInitV3: CopyType=0: use virtual address directly. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.653 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=4, taskResHead_=0, taskResTail_=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.667 [memory_task.cc:1007] 619792 ConstructPcieDmaSqe: stream_id=2, task_id=4, copyType=0, src=0xaaaaf2c1f4d0, dst=0x12c2c0000000, len=0x6c [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.679 [memory_task.cc:1166] 619792 ConstructSqeForMemcpyAsyncTask: MemcpyAsyncTask, stream_id=2, task_id=4, copyType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.691 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), head=0, tail=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.719 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=4, type=5(MEMCPY_ASYNC), head=0, tail=5, sqe_num=1, lastTail=4, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.733 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=4, task_type=5(MEMCPY_ASYNC). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.747 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.765 [callback.cpp:97]619792 aclrtLaunchCallback: start to execute aclrtLaunchCallback. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.790 [task_res.cc:380] 619792 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=2, taskResId=5, taskResHead_=0, taskResTail_=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.805 [task_info.cc:521] 619792 ConstructSqeForCallbackLaunchTask: CallbackLaunch,topic_type=4,stream_id=2,task_id=5,eventId=65535,isBlock=0,pid=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.816 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), head=0, tail=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.828 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=2, task_id=5, type=72(HOSTFUNC_CALLBACK), head=0, tail=6, sqe_num=1, lastTail=5, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.841 [task_submit.cc:573] 619792 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=2, cq_id=2, stream_id=2, task_id=5, task_type=72(HOSTFUNC_CALLBACK). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.855 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.867 [callback.cpp:109]619792 aclrtLaunchCallback: successfully execute aclrtLaunchCallback [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.876 [npu_driver.cc:3844] 619588 CqReportGet: cq report get start, device_id=2, tsId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.888 [npu_driver.cc:3851] 619588 CqReportGet: halCqReportGet count=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.899 [api_impl.cc:5087] 619588 ProcessReport: report[0], sqid=12291, streamId=2, taskId=5, eventId=65535, isBlock=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.913 [callback.cpp:138]619588 aclrtProcessReport: successfully aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.926 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.934 [device.cpp:231]619792 aclrtGetDevice: start to execute aclrtGetDevice [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.937 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.950 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.394.962 [acl.cpp:802]619792 aclGetRecentErrMsg: start to execute aclGetRecentErrMsg. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.394.988 [npu_driver.cc:404] 619792 StreamIdAlloc: stream id alloc begin, device_id=2, priority=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.014 [stream.cc:2246] 619792 AllocExecutedTimesSvm: stream_id=8. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.048 [stream_sqcq_manage.cc:147] 619792 Alloc: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.063 [npu_driver.cc:882] 619792 NormalSqCqAllocate: Set cq dep to 2048, sysMode=0 addrMode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.295 [npu_driver.cc:6125] 619792 LogicCqAllocateV2: cqeDepth=4096, isDvppGrp=0, inFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.349 [npu_driver.cc:6137] 619792 LogicCqAllocateV2: success: deviceId=2, tsId=0, cqId=1012, streamId=8, logicCq=1012, cqeDepth=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.370 [npu_driver.cc:6189] 619792 StreamBindLogicCq: success: deviceId=2, tsId=0, streamId=8, logicCq=1012. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.384 [npu_driver.cc:5916] 619792 GetSqRegVirtualAddrBySqid: addr high 32 bit=dfff, addr low 32 bit=e0144000, addr=dfffe0144000size=4096 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.397 [stream.cc:653] 619792 Setup: Success to get sq=8 sq reg virtual addr length=4096. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.434 [stream.cc:276] 619792 CreateStreamTaskRes: Normal STREAM Setup stream_id=8 flags_=0x0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.448 [stream.cc:688] 619792 Setup: stream setup end, stream_id=8, IsTaskSink=0, sqId=8, cqId=8, deviceId=2, streamResId=65534 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.395.461 [acl_resource_manager.cpp:814]619792 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.395.472 [acl_resource_manager.cpp:816]619792 HandleReleaseSourceByStream: it's create stream callback, currently do nothing. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.513 [engine.cc:188] 619792 SubmitTaskNormal: stream_id=8,task_id=0,task_type=87 (GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.530 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=8, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.544 [task_info.cc:409] 619792 ConstructSqeForGetDevMsgTask: stream_id:8, task_id:0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.554 [stream.cc:1400] 619792 AddTaskToList: public task, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.566 [stream.cc:2595] 619792 StarsAddTaskToStream: single-operator flow stream, stream_id=8, task_id=0, type=87(GET_DEVICE_MSG), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.581 [npu_driver.cc:751] 619792 SqTaskSend: device_id=2, ts_id=0, sq_id=8, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.591 [stars_engine.cc:893] 619792 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=8, cq_id=8, stream_id=8, task_id=0, task_type=87(GET_DEVICE_MSG). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.603 [stream.cc:1494] 619792 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.615 [npu_driver.cc:5547] 619792 GetSqHead: deviceId=2, tsId=0, sqId=8, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.627 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.638 [task.cc:185] 619792 GetTask: this stream no taskRes, stream_id=8 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.649 [logger.cc:152] 619792 TaskFinished: device_id=2, stream_id=8, task_id=0, task_type=87 (GET_DEVICE_MSG), task_finish_num=8 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.664 [stars_engine.cc:437] 619792 FinishedTaskReclaim: recycle task, stream_id=8, limited=0, taskHead=0, sqHead=1, endTaskId=0, recycleTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.676 [stream.cc:2886] 619792 StarsWaitForTask: No.0 stream_id=8, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.688 [stream.cc:852] 619792 TearDown: stream_id=8, sqId=8, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.700 [stream.cc:938] 619792 TearDown: stream_id=8, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.712 [stream.cc:1507] 619792 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.739 [stream_sqcq_manage.cc:451] 619792 FreeLogicCq: Return(0), threadIdentifier(281469230182304), devId(2), tsId(0), cqId(1012), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.759 [npu_driver.cc:6163] 619792 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=8, logicCq=1012. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.787 [npu_driver.cc:1180] 619792 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1012, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.799 [stream.cc:237] 619792 FreeStreamId: Free stream_id=8. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.810 [stream_sqcq_manage.cc:216] 619792 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.395.821 [npu_driver.cc:966] 619792 NormalSqCqFree: Normal sqcq free, sqId=8, cqId=8, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.261 [npu_driver.cc:974] 619792 NormalSqCqFree: Normal sqcq free success, sqId=8, cqId=8, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.278 [stream_sqcq_manage.cc:228] 619792 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=8, cq_id=8, stream_id=8, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.293 [npu_driver.cc:432] 619792 StreamIdFree: stream id free begin, device_id=2, stream_id=8, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.309 [npu_driver.cc:441] 619792 StreamIdFree: Stream id free success, stream_id=8, device_id=2, tsId=0, streamFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.323 [device_msg_handler.cc:110] 619792 HandleMsg: Begin to process device errMessage. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.613 [device_msg_handler.cc:127] 619792 HandleMsg: Finished to process device errInfo, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.644 [runtime.cc:5239] 619792 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.396.662 [npu_driver.cc:500] 619792 GetFaultEvent: get fault event, device_id=2, filterFlag=8, outputLen=128. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.396.714 [acl.cpp:782]619792 GetFaultEventInfo: can not get fault event of device 2, runtime errorCode is 0 [INFO] GE(619427,python3.9):2025-04-03-09:35:50.396.730 [error_manager.cc:989]619792 GetRawErrorMessages:current work_stream_id:0 [CRITICAL] KERNEL(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.397.020 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59] Launch] LCCL AllReduce failed. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.397.054 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:116] TraceGraphEval] Length of analysis graph stack is empty. [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.397.072 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:670] GetEvalStackInfo] Get graph analysis information begin [INFO] DEBUG(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.397.085 [mindspore/ccsrc/pipeline/jit/ps/debug/trace.cc:673] GetEvalStackInfo] Length of analysis information stack is empty. [INFO] RUNTIME_FRAMEWORK(619427,fffea97aefa0,python3.9):2025-04-03-09:35:50.397.151 [mindspore/ccsrc/runtime/pipeline/async_rqueue.cc:133] WorkerLoop] Run task failed, error msg:LCCL AllReduce failed. ---------------------------------------------------- - C++ Call Stack: (For framework developers) ---------------------------------------------------- mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:50.424.033 [awatchdog_monitor.c:160](tid:619463) thread 619758 not exist [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.436.178 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.436.217 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.436.230 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.436.243 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.436.259 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.436.273 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.436.288 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.171 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.177 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.176 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.181 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.183 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.468.188 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.280 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.298 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.314 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.319 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.325 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.332 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.337 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.345 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.347 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.351 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.355 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.365 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.368 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.381 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.392 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.394 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.400 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.403 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.412 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.415 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.416 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.423 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.428 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.431 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.434 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.440 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.445 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.450 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.451 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.461 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.478 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.483 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.501 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.468.518 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.534 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.468.551 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.916.182 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.916.228 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.916.247 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.916.268 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.916.323 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.916.339 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.916.376 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.948.186 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:50.948.224 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.948.239 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:50.948.256 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.948.275 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.948.292 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.948.308 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.977.716 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.758 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.977.780 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.977.822 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.843 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.867 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.940 [stars_engine.cc:643] 619427 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=0, stream_id=2, task_id=1, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.972 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=0, task_type=5 (MEMCPY_ASYNC), task_finish_num=9 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.977.988 [task_manager.cc:485] 619427 TaskFailCallBack: task ok, stream_id=2, task_id=0, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.005 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.022 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=1, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=10 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.035 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=1, taskDesHeadIdx=1, taskResHead_=2, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.066 [stars_engine.cc:643] 619427 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=1, stream_id=2, task_id=3, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.081 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=2, task_type=5 (MEMCPY_ASYNC), task_finish_num=11 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.093 [task_manager.cc:485] 619427 TaskFailCallBack: task ok, stream_id=2, task_id=2, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.104 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=2, taskDesHeadIdx=2, taskResHead_=3, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.117 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=3, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=12 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.128 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=3, taskDesHeadIdx=3, taskResHead_=4, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.145 [stars_engine.cc:643] 619427 ProcLogicCqUntilEmpty: Notify: cnt=3, idx=2, stream_id=2, task_id=5, sqeType=1, retCode=0, errType=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.156 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=4, task_type=5 (MEMCPY_ASYNC), task_finish_num=13 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.167 [task_manager.cc:485] 619427 TaskFailCallBack: task ok, stream_id=2, task_id=4, retCode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.178 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=4, taskDesHeadIdx=4, taskResHead_=5, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.191 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=2, task_id=5, task_type=72 (HOSTFUNC_CALLBACK), task_finish_num=14 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.204 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=2, taskId=5, taskDesHeadIdx=5, taskResHead_=6, tail=6, taskResTail_=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.226 [stars_engine.cc:622] 619427 ProcLogicCqUntilEmpty: Task Wait: stream_id=2, logicCqId=1006, retCode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.240 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.254 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.978.265 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.978.286 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.305 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.318 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.330 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.342 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.354 [stream.cc:2886] 619427 StarsWaitForTask: No.1 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.366 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.378 [stream.cc:2886] 619427 StarsWaitForTask: No.2 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.394 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.407 [stream.cc:2886] 619427 StarsWaitForTask: No.3 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.419 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.430 [stream.cc:2886] 619427 StarsWaitForTask: No.4 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.442 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.453 [stream.cc:2886] 619427 StarsWaitForTask: No.5 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.463 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.473 [stream.cc:2886] 619427 StarsWaitForTask: No.6 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.485 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.495 [stream.cc:2886] 619427 StarsWaitForTask: No.7 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.506 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.517 [stream.cc:2886] 619427 StarsWaitForTask: No.8 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.527 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.537 [stream.cc:2886] 619427 StarsWaitForTask: No.9 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.548 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.558 [stream.cc:2886] 619427 StarsWaitForTask: No.10 stream_id=3, taskId=65535, currentId=65535 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.573 [event.cc:310] 619427 InitEventAllocFlag: Stars not need to alloc id in this flag=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.602 [task_res.cc:380] 619427 AllocTaskInfoByTaskResId: alloc taskinfo success, device_id=2, stream_id=3, taskResId=0, taskResHead_=0, taskResTail_=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.628 [event.cc:249] 619427 UpdateLatestRecord: device_id=2, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.642 [event_task.cc:259] 619427 ConstructSqeForEventRecordTask: event_record: device_id=2, stream_id=3, task_id=0, event_id=4294967295, sq_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.656 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), head=0, tail=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.667 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=3, task_id=0, type=2(EVENT_RECORD), head=0, tail=1, sqe_num=1, lastTail=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.686 [task_submit.cc:573] 619427 AllocTaskAndSendStars: device_id=2, ts_id=0, sq_id=3, cq_id=3, stream_id=3, task_id=0, task_type=2(EVENT_RECORD). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.700 [stars_engine.cc:1122] 619427 SyncTask: Task Wait: device_id=2, stream_id=3, task_id=0, logicCqId=1007. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.732 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.757 [stars_engine.cc:1071] 619427 IsSyncTaskFinish: Sync task finished, sqHead=1 finishedTaskId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.773 [stars_engine.cc:1017] 619427 ProcReport: Get logic report: cnt=1, idx=0, stream_id=3, report task_id=0, sync task_id=0, sqeType=3, sqeHead=0, retCode=0, errType=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.789 [stars_engine.cc:1027] 619427 ProcReport: taskType=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.799 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=3, task_id=0, task_type=2 (EVENT_RECORD), task_finish_num=15 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.811 [event.cc:249] 619427 UpdateLatestRecord: device_id=2, event_id=-1, cur stream_id=3, cur task_id=0, latest stream_id=3, latest task_id=0, status=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.824 [event_task.cc:211] 619427 DoCompleteSuccessForEventRecordTask: event_record: device_id=2, stream_id=3, task_id=0, sq_id=3, event_id=4294967295, timeline=0xffffffffffffffff, timestamp=0x1ea5714507e [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.838 [event.cc:122] 619427 TryFreeEventIdAndCheckCanBeDelete: device_id=2, event_id=-1, map_count=0, isNewMode=0, isIdFromDrv=0, newestId=-1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.854 [task_res.cc:323] 619427 RecycleTaskInfoO1: recycleO1 taskinfo success, device_id=2, stream_id=3, taskId=0, taskDesHeadIdx=0, taskResHead_=1, tail=1, taskResTail_=1 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.874 [npu_driver.cc:5618] 619427 GetSqEnable: deviceId=2, tsId=0, sqId=3, enable=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.885 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.896 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.978.906 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.978.981 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.978.998 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.011 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.023 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.035 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.048 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.060 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.070 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.081 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.093 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.104 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.115 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.140 [stars_engine.cc:622] 619427 ProcLogicCqUntilEmpty: Task Wait: stream_id=3, logicCqId=1007, retCode=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.158 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:50.979.171 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.182 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:50.979.196 [memory.cpp:421]619427 aclrtMemcpy: start to execute aclrtMemcpy, destMaxSize = 108, srcSize = 108, kind = 2 F ========================================================================================================================= FAILURES ========================================================================================================================= ______________________________________________________________________________________________________________________ test_AllReduce ______________________________________________________________________________________________________________________ def test_AllReduce(): """ Feature: lccl operator test. Description: msrun lccl all_reduce 8P case. Expectation: success """ all_reduce = Net() output = all_reduce() expect0 = np.ones([3, 1, 3, 3]).astype(np.float32) * 0 for i in range(size): part = np.ones([3, 1, 3, 3]).astype(np.float32) * 0.01 * (i + 1) expect0 += part > diff0 = output[0].asnumpy() - expect0 test_lccl_allreduce.py:66: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Tensor(shape=[3, 1, 3, 3], dtype=Float32, value= [[[[ -nan, -nan, -nan], [ ...an], [ -nan, -nan, -nan], [ -nan, -nan, -nan]]]]) def asnumpy(self): """ Convert tensor to numpy array. Returns self tensor as a NumPy ndarray. This tensor and the returned ndarray share the same underlying storage. Changes to self tensor will be reflected in the ndarray. Returns: A numpy ndarray which shares the same underlying storage with the tensor. Examples: >>> from mindspore import Tensor >>> import numpy as np >>> x = Tensor(np.array([1, 2], dtype=np.float32)) >>> y = x.asnumpy() >>> y[0] = 11 >>> print(x) [11. 2.] >>> print(y) [11. 2.] """ if self.has_init: self.init_data() > return TensorPy_.asnumpy(self) E RuntimeError: LCCL AllReduce failed. E E ---------------------------------------------------- E - C++ Call Stack: (For framework developers) E ---------------------------------------------------- E mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hcom_all_reduce.cc:59 Launch /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/tensor.py:989: RuntimeError ===================================================================================================================== warnings summary ===================================================================================================================== ../../../../../../../../usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866 /usr/local/Ascend/ascend-toolkit/8.1.RC1/python/site-packages/tbe/tvm/contrib/ccec.py:866: DeprecationWarning: invalid escape sequence \L if not dirpath.find("AppData\Local\Temp"): ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/classifier/transdata/transdata_classifier.py:222: DeprecationWarning: invalid escape sequence \B """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:143: DeprecationWarning: invalid escape sequence \c """ ../../../../../../../../usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170 /usr/local/Ascend/ascend-toolkit/latest/python/site-packages/tbe/dsl/unify_schedule/vector/transdata/common/graph/transdata_graph_info.py:170: DeprecationWarning: invalid escape sequence \c """ ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero. setattr(self, word, getattr(machar, word).flat[0]) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero. return self._float_to_str(self.smallest_subnormal) ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2.py:57: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad.py:56: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/batchnorm_fold2_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("batchnorm_fold2_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:51: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/correction_mul_grad.py:143: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("correction_mul_grad_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad.py:92: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perlayer_grad_reduce.py:49: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perlayer_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad.py:91: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_learned_scale_quant_perchannel_grad_reduce.py:48: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_learned_scale_quant_perchannel_grad_d_reduce") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel.py:52: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perchannel_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_perchannel_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer.py:54: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/fake_quant_perlayer_grad.py:81: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("fake_quant_per_layer_grad") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perchannel.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perchannel") ../../../../../../anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50 /home/jenkins/anaconda3/envs/ci39/lib/python3.9/site-packages/mindspore/ops/_op_impl/_custom_op/minmax_update_perlayer.py:50: DeprecationWarning: te_fusion.fusion_manager.fusion_manager.register is deprecated,please replace it with tbe.common.register.register_op_compute @fusion_manager.register("minmax_update_perlayer") -- Docs: https://docs.pytest.org/en/stable/warnings.html ================================================================================================================= short test summary info ================================================================================================================== FAILED test_lccl_allreduce.py::test_AllReduce - RuntimeError: LCCL AllReduce failed. ============================================================================================================= 1 failed, 26 warnings in 24.66s ============================================================================================================== 4 mem name: 000973ab00000000000003170400ffd00004 [2025-04-03 09:35:45.340838] [debug] [619758] [lcal_comm.cpp:515] rank 5 mem name: 000973af000000000000031c0500ffd40005 [2025-04-03 09:35:45.340845] [debug] [619758] [lcal_comm.cpp:515] rank 6 mem name: 000973b300000000000003190600ffd80006 [2025-04-03 09:35:45.340851] [debug] [619758] [lcal_comm.cpp:515] rank 7 mem name: 000973b7000000000000031a0700ffdc0007 [2025-04-03 09:35:45.340859] [debug] [619758] [lcal_comm.cpp:517] AllGather: Get other rank mem name [2025-04-03 09:35:45.360581] [debug] [619758] [lcal_comm.cpp:254] InitCommMem 2/8, localRank_ : 2, localRankSize_ : 8 success [2025-04-03 09:35:45.360732] [info] [619758] [lcal_comm.cpp:259] LcalCommInit 2/8 success and extraFlag:0 commArgs_.localRank : 2 commArgs_.localRankSize : 8 [2025-04-03 09:35:50.387973] [error] [619792] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.392261] [error] [619792] [lccl.cpp:207] Lccl not support. [2025-04-03 09:35:50.394914] [error] [619792] [lccl.cpp:207] Lccl not support. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.169 [mindspore/ccsrc/pybind_api/init.cc:651] operator()] Start register... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.217 [mindspore/ccsrc/pybind_api/init.cc:654] operator()] Start mindspore.profiler... [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:50.988.298 [mindspore/profiler/envprofiler.py:57] analyse start [INFO] ME(619427:281473076260896,MainProcess):2025-04-03-09:35:50.988.451 [mindspore/profiler/envprofiler.py:59] Profiler is not initialized, skip analyse. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.524 [mindspore/ccsrc/pybind_api/init.cc:661] operator()] Start EmbeddingCacheScheduler... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.599 [mindspore/ccsrc/pybind_api/init.cc:668] operator()] Start releasing dataset handles... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.641 [mindspore/ccsrc/pybind_api/init.cc:671] operator()] End release dataset handles. [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:50.988.667 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2616] FinalizeCluster] Start finalize the cluster instance. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.150 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:132] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.254 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619427,ffff2e241fa0,python3.9):2025-04-03-09:35:51.007.350 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.436 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.455 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619427,ffff2ea51fa0,python3.9):2025-04-03-09:35:51.007.512 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.583 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.599 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.655 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:563] Finalize] Delete send event loop [INFO] DISTRIBUTED(619427,ffff2d221fa0,python3.9):2025-04-03-09:35:51.007.750 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.882 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.007.900 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:570] Finalize] Delete recv event loop [INFO] DISTRIBUTED(619427,ffff2da31fa0,python3.9):2025-04-03-09:35:51.007.939 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.024 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:210] Finalize] Stop loop succ [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.042 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:584] Finalize] Delete connection pool. [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.063 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2619] FinalizeCluster] End finalize the cluster instance. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.091 [mindspore/ccsrc/pybind_api/resource/manager.cc:259] ClearResAtexit] Pipeline clear all resource [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.233 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.258 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.275 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.310 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.327 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.344 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.359 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.374 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.386 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.401 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.415 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.428 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.440 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.450 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.461 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.483 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.500 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.513 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=2, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.526 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=2, taskId=5, currentId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.538 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=2, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.549 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.560 [stream.cpp:131]619427 aclrtSynchronizeStreamWithTimeout: start to execute aclrtSynchronizeStreamWithTimeout, timeout = -1ms [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.573 [api_impl.cc:1394] 619427 StreamSynchronize: stream_id=3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.582 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=3, head=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.594 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=3, taskId=0, currentId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.008.606 [api_error.cc:895] 619427 StreamSynchronize: stream synchronize exit, stream_id=3, result=0x0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.008.616 [stream.cpp:146]619427 aclrtSynchronizeStreamWithTimeout: Synchronize stream with timeout success [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.665 [mindspore/ccsrc/pybind_api/resource/manager.cc:82] RecordExitStatus] Status record: system exit. [INFO] DEBUG(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.730 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.786 [mindspore/core/mindrt/src/actor/actormgr.cc:165] Finalize] mindrt Actors finish exiting. [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.801 [mindspore/core/mindrt/src/actor/actormgr.cc:168] Finalize] mindrt Threads finish exiting. [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.816 [mindspore/core/mindrt/src/actor/actormgr.cc:179] Finalize] mindrt IOMGRS finish exiting. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.008.997 [mindspore/ccsrc/pybind_api/resource/manager.cc:134] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.017 [mindspore/ccsrc/pybind_api/resource/manager.cc:136] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.790 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.816 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1000] ClearInfo] Clean graph resource! [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.833 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:995] ClearRes] Clean Graph executor resource! [INFO] PIPELINE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline_jit.cc:426] ClearRes] Clean JIT executor resource! [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.009.865 [mindspore/ccsrc/pybind_api/resource/manager.cc:155] ClearResPart2] Start clear PyNativeExecutor... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.153 [mindspore/ccsrc/pybind_api/resource/manager.cc:157] ClearResPart2] End clear PyNativeExecutor. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.175 [mindspore/ccsrc/pybind_api/resource/manager.cc:159] ClearResPart2] Start clear ConfigManager... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.190 [mindspore/ccsrc/pybind_api/resource/manager.cc:161] ClearResPart2] End clear ConfigManager. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.257 [mindspore/ccsrc/common/thread_pool.cc:41] ThreadPool] Set max_thread_num_ to 4 [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.275 [mindspore/ccsrc/pybind_api/resource/manager.cc:165] ClearResPart2] Start clear device context... [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.010.291 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device Ascend_2 [INFO] GE_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.018.151 [mindspore/ccsrc/backend/ge_backend/graph_ir/df_graph_manager.cc:154] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.018.189 [mindspore/ccsrc/backend/ge_backend/executor/ge_graph_executor.cc:1440] UnregisterExternalAllocator] The graph_runner is not exist [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.018.213 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.226 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.245 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.260 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.283 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.296 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.307 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.334 [device.cpp:143]619427 aclrtSetDevice: start to execute aclrtSetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.354 [api_impl.cc:2999] 619427 SetDevice: device_id=2, deviceMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.366 [device.cc:306] 619427 Init: device mode has been already set, mode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.381 [runtime.cc:3393] 619427 PrimaryContextRetain: Context inc ref, ts_id=0, count=0xb [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.397 [runtime.cc:3560] 619427 MsProfNotifyWhenSetDevice: MsprofNotifySetDevice success, devId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.410 [api_impl.cc:2987] 619427 NewDevice: SetCurRef = 0xaaaae7011ab0,userDeviceId=2,deviceId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.423 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.444 [api_impl.cc:2992] 619427 NewDevice: New device success, user_device_id=2, die_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.018.459 [api_impl.cc:3025] 619427 SetDevice: SetDevice success, curCtx=0xaaaae7011ab0, user_device_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.473 [acl.cpp:115]619427 InitSocVersion: get SocVersion success, SocVersion = Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.487 [acl.cpp:121]619427 GetSocVersion: socVersion is Ascend910B3 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.018.505 [device.cpp:154]619427 aclrtSetDevice: successfully execute aclrtSetDevice, deviceId = 2 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:51.018.529 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:46] DestroyHandlerByName] Destroy watch dog thread by group name: hccl_world_group [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:51.192.867 [awatchdog_monitor.c:160](tid:619463) thread 619792 not exist [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.184 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.184 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.188 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.188 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.196 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.294 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.317 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.318 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.300.316 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.340 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.345 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.348 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.366 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.371 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.377 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.380 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.384 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.394 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.396 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.400 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.405 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.408 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.411 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.416 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.425 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.439 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.440 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.448 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.451 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.458 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.462 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.460 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.469 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.479 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.485 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.527 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.529 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.541 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.555 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.300.573 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.590 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.300.608 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.428.186 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.428.221 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.428.238 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.428.256 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.428.276 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.428.291 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.428.310 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.460.170 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.460.198 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.460.214 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.460.229 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.460.243 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.460.257 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.460.270 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.532.737 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:51.532.770 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:51.532.815 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:51.532.832 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.532.856 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.532.928 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.940.168 [trs_cb_event.c:119][ascend][curpid:619427,619588][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =576; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.940.201 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.940.219 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.940.234 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.940.246 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.940.256 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.940.271 [npu_driver.cc:3828] 619588 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=576. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.972.176 [trs_cb_event.c:119][ascend][curpid:619427,619589][drv][tsdrv][trsCbEventWait]Call halEschedWaitEvent warn. (devId=2, tid =577; ret=16) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:51.972.204 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.972.219 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:51.972.235 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.972.249 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:51.972.265 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:51.972.281 [npu_driver.cc:3828] 619589 CqReportIrqWait: Irq wait start, deviceId=2, tsId=0, groupId=577. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.174 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.179 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.182 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.185 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.183 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.132.187 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.266 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.280 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.295 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.296 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.311 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.317 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.325 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.325 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.329 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.340 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.343 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.350 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.352 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.358 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.362 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.375 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.380 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.387 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.392 [tensor_data_transfer.cpp:1054]619635 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is _npu_log [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.394 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.398 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.411 [tensor_data_transfer.cpp:1056]619635 acltdtReceiveTensor: new process, use queue process [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.417 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.425 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.432 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.440 [api_impl.cc:6121] 619635 MemQueuePeek: Start to peek queue on devId 2, qid is 0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.447 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.461 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.490 [npu_driver.cc:4330] 619635 MemQueuePeek: peek mem queue, deviceId=2, qid=0, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.498 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.507 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.132.525 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.537 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.547 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.556 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.132.566 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] DEVICE(619427,fffe8fffefa0,python3.9):2025-04-03-09:35:52.340.363 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:177] WatchDogProcess] end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.461 [op_base.cc:2136] [619427]Entry-HcclCommDestroy: op_base comm destroy begin [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.340.515 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.547 [adapter_rts.cc:198] [619427][hrtGetDeviceRefresh]deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.602 [op_base.cc:2151] [619427][HcclCommDestroy] comm state is IDLE [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.632 [op_base.cc:128] [619427][HcclGetDeviceId] deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.669 [op_base.cc:2168] [619427]com is not global com [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.694 [hcom_plugin.cc:381] [619427][Unset][GroupTopoInfo]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.754 [hccl_comm.cc:1375] [619427]UnRegistTaskAbortHandler begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.776 [task_abort_handler.cc:140] [619427]TaskAbortHandler::DeInit commVector size is [1], ref_ count is [1] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.340.794 [api_impl.cc:4192] 619427 SetTaskAbortCallBack: set task abort callback. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.340.808 [runtime.cc:4698] 619427 SetTaskAbortCallBack: SetTaskAbortCallBack finish, moduleName:HCCL. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.823 [hccl_communicator.cc:2112] [619427][HcclCommunicator][DestroyAicpuComm]start to destroy the aicpu comm, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.340.839 [hccl_communicator.cc:2114] [619427][HcclCommunicator][DestroyAicpuComm]Device type is not A3, no needs to destroy the aicpu comm. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.854 [hccl_communicator.cc:2082] [619427]start to stop the backGround Thread [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.340.870 [heartbeat.cc:1203] [619427]Heartbeat has been destroyed [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.896 [heartbeat.cc:451] [619427][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.340.913 [heartbeat.cc:452] [619427]Heartbeat has been destroyed [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.942 [hccl_impl.cc:69] [619427]start hccl resources destruction:deviceLogicId[2] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.340.977 [heartbeat.cc:1203] [619427]Heartbeat has been destroyed [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.340.994 [heartbeat.cc:451] [619427][UnRegisterRanks]group[192.168.0.44%enp67s0f5_60008_0_1743644144960128] UnregisterRanks Start. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.341.007 [heartbeat.cc:452] [619427]Heartbeat has been destroyed [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.341.340 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280003000 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.341.682 [task_exception_handler.cc:1232] [619427]deInit taskFailCallback [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.342.654 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c2800a5000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.342.680 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280004000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.342.744 [api_error.cc:1278] 619427 HostFree: Free host memory succeed, host addr=0x12c0807f0000. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.342.765 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280146000 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.342.799 [hccl_socket_manager.cc:81] [619427][DeInit][Server]ip[192.168.2.197] port[16666] serverSocketRef.Count() = 0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.342.819 [hccl_socket_manager.cc:83] [619427][DeInit][Server]ip[192.168.2.197] port[16666] [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.342.835 [hccl_socket.cc:232] [619427][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.342.848 [network_manager.cc:69] [619427]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.342.882 [network_manager.cc:458] [619427]NetworkManager devicePhyId_[2] vnic stopped. refcount[0] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:52.342.920 [ra_host.c:911]tid:619427,ra_socket_listen_stop(911) : Input parameters: [0]th, phy_id[2], local_ip[192.168.2.197] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:52.343.334 [ra_host.c:404]tid:619427,ra_socket_deinit(404) : Input parameters: phy_id[2] family[2] local_ip[192.168.2.197] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.343.501 [hccl_socket.cc:68] [619427][HcclSocket][DeInit] device[2] stops listen on ip[192.168.2.197], port[16666] success, socketType[0]. [WARNING] HCCL(619427,python3.9):2025-04-03-09:35:52.343.525 [hccl_socket.cc:232] [619427][Close] socket's fdHandle is null, do not need close. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.343.548 [hccl_communicator.cc:1741] [619427][HcclCommunicator][ReleasePreemptSocket] release all preempt socket success, comm id[192.168.0.44%enp67s0f5_60008_0_1743644144960128]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.343.564 [network_manager.cc:69] [619427]NetworkManager::GetInstance deviceLogicID[2]. [INFO] HCCP(619427,python3.9):2025-04-03-09:35:52.343.579 [ra_host.c:248]tid:619427,ra_deinit(248) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(619427,python3.9):2025-04-03-09:35:52.343.599 [ra_hdc.c:651]tid:619427,ra_hdc_deinit(651) : hdc deinit start! phy_id[2] [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.343.701 [hdc_client.c:417][ascend][curpid:619427,619427][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4275) [INFO] ROCE(619427,python3.9):2025-04-03-09:35:52.343.793 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] HCCP(619427,python3.9):2025-04-03-09:35:52.343.812 [ra_hdc.c:685]tid:619427,ra_hdc_deinit(685) : hdc deinit OK! phy_id[2] [INFO] ROCE(619427,python3.9):2025-04-03-09:35:52.343.843 [dl_hal_function.c:208]dl_hal_deinit(208) : dl_hal_deinit success! [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.907 [tsd_client.cpp:189][TsdCapabilityGet][tid:619427] TsdCapabilityGet Begin. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.925 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.940 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.955 [process_mode_manager.cpp:1042][CapabilityGet][tid:619427] [ProcessModeManager] enter into CapabilityGet process deviceId[2] type[6] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.343.971 [adapter_tdt.cc:66] [619427]Get TsdCapability success. deviceLogicId[2] [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.985 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.343.995 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.344.009 [process_mode_manager.cpp:1735][ProcessCloseSubProcList][tid:619427] enter ExcuteClosePidList cnt:1, tsdSupportLevel_:63 [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.344.025 [process_mode_manager.cpp:1794][ExcuteClosePidList][tid:619427] add close subproc:6167, proctype:0 [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.344.063 [hdc_client.cpp:154][TsdRecvData][tid:619427] recv timeout is:150000 ms [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.416.593 [process_mode_msg_parse.cpp:115][ServerToClientMsgProc][tid:619427] [ServerToClientMsgProc] [sessionID=1] [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.416.617 [process_mode_msg_parse.cpp:110][DeviceMsgProcess][tid:619427] [TsdClient] DeviceMsgProc recvMsg realDeviceId[2] msgType[61] localDevId[2] rspCode[0]heterogeneousSubPid[6167], tsdSupportLevel_[63] [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.416.638 [hdc_client.cpp:172][TsdRecvData][tid:619427] [deviceId=2] the Tsd recv data already [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.416.652 [process_mode_manager.cpp:1809][ExcuteClosePidList][tid:619427] leave ExcuteClosePidList startindex:0, cnt:1 [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.416.667 [process_mode_manager.cpp:1769][ProcessCloseSubProcList][tid:619427] ExcuteClosePidList success reserveCnt:1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.680 [adapter_tdt.cc:52] [619427]Open TsdProcess success. deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.692 [network_manager.cc:405] [619427][DeInit]finish CloseHccpProcess, phy_id[2], hdc_type[18], nicDeployment[1], hasBackup[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.705 [network_manager.cc:409] [619427]NetworkManager: deinit nic success, nicPosition[1] ref[0]. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.720 [hccl_communicator_attrs.cc:1307] [619427][HcclCommunicator]IsSupportEnableRoce [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.731 [hccl_communicator_attrs.cc:1275] [619427]IsEnableRoce ret: 0, interServer_: 0, isInterServerVnic: 0, roceSwitch: 0, isSingleMeshAggregation_: 1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.751 [notify_pool_impl.cc:43] [619427]NotifyPoolImpl Destroy. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.766 [notify_pool_impl.cc:56] [619427]NotifyPoolImpl Destroy success. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.796 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.813 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.827 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.853 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.868 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.878 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.891 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.908 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.924 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=0 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.416.937 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.948 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=0. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.416.961 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=0.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.977 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.416.988 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.001 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.016 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.029 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.041 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=1 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.055 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.066 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=1. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.077 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=1.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.092 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.101 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.115 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.125 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.141 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.151 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.163 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.178 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.188 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.201 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=3 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.213 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.235 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=3. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.250 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=3.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.267 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.279 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.292 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.303 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.316 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.329 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=4 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.343 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.357 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=4. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.369 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=4.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.385 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.395 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.406 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.415 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.428 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.442 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=5 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.456 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.468 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=5. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.484 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=5.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.497 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.507 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.517 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.529 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.542 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.555 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=6 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.569 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.584 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=6. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.601 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=6.) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.623 [api_impl.cc:3100] 619427 GetDevicePhyIdByIndex: get PhyId by Index=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.634 [api_impl.cc:3111] 619427 GetDevicePhyIdByIndex: GetDevicePhyIdByIndex, device_mode=0, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.645 [api_impl.cc:3130] 619427 GetDevicePhyIdByIndex: cloud v2, device_mode=0, device_id=2 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.655 [p2p_mgmt.cc:171] [619427]local logic id:2, local physic id:2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.666 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.684 [api_impl.cc:3520] 619427 GetPhyDeviceInfo: get phy device info, phyId=7 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.697 [p2p_mgmt.cc:178] [619427]disable p2p: local logic id:2, remote physic id:7. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.715 [api_impl.cc:3184] 619427 DisableP2P: Disable P2P devIdDes=2, phyIdSrc=7. [INFO] DRV(619427,python3.9):2025-04-03-09:35:52.417.733 [devdrv_pcie.c:890][ascend][curpid:619427,619427][drv][devmng][drvP2PAttrOp]Disable P2P. (logic_devid=2; peer_phy_devid=7.) [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.751 [hccl_communicator.cc:2043] [619427]Profiling is deinitiated. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.769 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280000000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.789 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280001000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.417.807 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280002000 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.829 [opexecounter.cc:103] [619427]free counter mem resource [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.417.846 [atrace_api.c:118](tid:619427) AtraceDestroy start. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.417.864 [tracer_mgr_operate.c:248](tid:619427) destroy object HCCL_619758_2, exitSave(true). [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.417.880 [atrace_api.c:120](tid:619427) AtraceDestroy end. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.417.894 [adapter_trace.cc:33] [619427]Call TraceDestroy, Params:handle[0xaaaad6499b78] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.420.433 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c280147000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.420.490 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c100c00000 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.523 [queue_notify_manager.cc:100] [619427]QueueNotifyManager Destroy. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.537 [queue_notify_manager.cc:106] [619427]QueueNotifyManager Destroy success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.836 [queue_notify_manager.cc:100] [619427]QueueNotifyManager Destroy. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.855 [queue_notify_manager.cc:106] [619427]QueueNotifyManager Destroy success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.964 [ccl_buffer_manager.cc:125] [619427][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.983 [ccl_buffer_manager.cc:147] [619427][HCCL_TRACE][ReleaseCCLbuffer]CCLBuffer is null, no need to release. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.420.998 [ccl_buffer_manager.cc:160] [619427][HCCL_TRACE][ReleaseAIVbuffer]Release inAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.421.012 [ccl_buffer_manager.cc:163] [619427][HCCL_TRACE][ReleaseAIVbuffer]Release outAIVbuffer. buffer ptr[(nil)], size[0] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.421.027 [ccl_buffer_manager.cc:166] [619427][HCCL_TRACE][ReleaseAIVbuffer]Release AIV buffer success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.421.092 [op_base.cc:128] [619427][HcclGetDeviceId] deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.421.126 [op_base.cc:2196] [619427]op_base comm destroy complete, take time [80622]us, group[192.168.0.44%enp67s0f5_60008_0_1743644144960128], deviceLogicId[2]. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.421.150 [device.cpp:183]619427 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.175 [api_impl.cc:3419] 619427 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.190 [runtime.cc:3444] 619427 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0xa, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.203 [api_impl.cc:3449] 619427 DeviceReset: Succ, userDeviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.421.216 [device.cpp:189]619427 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 2. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.421.279 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:364] FinalizeHccl] Start destroy hccl adapter for GE_MODE [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.421.314 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:806] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.421.329 [hcom_executor.cc:40] [619427]Hcom Excutor Finalize start. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.381 [api_impl.cc:3419] 619647 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.383 [api_impl.cc:3419] 619649 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.414 [runtime.cc:3444] 619647 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x9, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.426 [runtime.cc:3444] 619649 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x8, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.431 [api_impl.cc:3449] 619647 DeviceReset: Succ, userDeviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.442 [api_impl.cc:3449] 619649 DeviceReset: Succ, userDeviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.835 [api_impl.cc:3419] 619650 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.862 [runtime.cc:3444] 619650 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x7, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.880 [api_impl.cc:3449] 619650 DeviceReset: Succ, userDeviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.421.981 [api_impl.cc:3419] 619648 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.020 [runtime.cc:3444] 619648 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x6, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.036 [api_impl.cc:3449] 619648 DeviceReset: Succ, userDeviceId=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.244 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.422.272 [adapter_rts.cc:321] [619427][hrtCtxSetCurrent] success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.422.286 [stream.cc:182] [619427]Switch Ctx ret[0], curCtx[0xaaaae7011ab0], setCtx[0xaaaae7011ab0], stream id[5] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.322 [api_impl.cc:1228] 619427 StreamDestroy: stream_id=5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.422.358 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.422.401 [single_op_manager.cc:50]619427 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf249b120 [INFO] GE(619427,python3.9):2025-04-03-09:35:52.422.449 [caching_allocator.cc:402]619427 TryFreeBlocks:Try free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.422.476 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:52.422.519 [npu_memory_allocator.cc:72]619427 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.422.537 [stream_executor.cpp:197]619427 RemoveExecutor: To remove executor by stream = 187651186077984 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.422.569 [acl_resource_manager.cpp:823]619427 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.601 [stream.cc:852] 619427 TearDown: stream_id=5, sqId=5, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.621 [stream.cc:938] 619427 TearDown: stream_id=5, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.634 [stream.cc:1507] 619427 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.689 [stream_sqcq_manage.cc:451] 619427 FreeLogicCq: Return(0), threadIdentifier(281473076260896), devId(2), tsId(0), cqId(1009), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.715 [npu_driver.cc:6163] 619427 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=5, logicCq=1009. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.422.749 [npu_driver.cc:1180] 619427 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1009, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.425.392 [stream.cc:237] 619427 FreeStreamId: Free stream_id=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.425.437 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.425.458 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=5, cqId=5, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.426.038 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=5, cqId=5, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.426.061 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=5, cq_id=5, stream_id=5, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.426.079 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=5, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.426.099 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=5, device_id=2, tsId=0, streamFlag=1536 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.427.866 [stream.cc:185] [619427][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaaf249b120], stream id[5] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.427.892 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.427.910 [adapter_rts.cc:321] [619427][hrtCtxSetCurrent] success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.427.925 [stream.cc:193] [619427]Restore Ctx ret[0], setCtx[0xaaaae7011ab0], stream id[5] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.427.988 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.428.010 [adapter_rts.cc:321] [619427][hrtCtxSetCurrent] success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.428.023 [stream.cc:182] [619427]Switch Ctx ret[0], curCtx[0xaaaae7011ab0], setCtx[0xaaaae7011ab0], stream id[4] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.069 [api_impl.cc:1228] 619427 StreamDestroy: stream_id=4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.428.089 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.428.119 [single_op_manager.cc:50]619427 ReleaseResource:ReleaseResource in. resource id = 0xaaaaf241f0c0 [INFO] GE(619427,python3.9):2025-04-03-09:35:52.428.135 [caching_allocator.cc:402]619427 TryFreeBlocks:Try free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.428.148 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:52.428.174 [npu_memory_allocator.cc:72]619427 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.428.191 [stream_executor.cpp:197]619427 RemoveExecutor: To remove executor by stream = 187651185569984 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.428.207 [acl_resource_manager.cpp:823]619427 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.226 [stream.cc:852] 619427 TearDown: stream_id=4, sqId=4, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.244 [stream.cc:938] 619427 TearDown: stream_id=4, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.260 [stream.cc:1507] 619427 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.283 [stream_sqcq_manage.cc:451] 619427 FreeLogicCq: Return(0), threadIdentifier(281473076260896), devId(2), tsId(0), cqId(1008), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.307 [npu_driver.cc:6163] 619427 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=4, logicCq=1008. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.428.344 [npu_driver.cc:1180] 619427 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1008, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.391 [stream.cc:237] 619427 FreeStreamId: Free stream_id=4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.414 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.431 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=4, cqId=4, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.881 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=4, cqId=4, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.903 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=4, cq_id=4, stream_id=4, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.921 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=4, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.430.938 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=4, device_id=2, tsId=0, streamFlag=1536 [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.432.784 [stream.cc:185] [619427][HCCL_TRACE]StreamDestroy, streamPtr[0xaaaaf241f0c0], stream id[4] [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.432.816 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.432.837 [adapter_rts.cc:321] [619427][hrtCtxSetCurrent] success [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.432.853 [stream.cc:193] [619427]Restore Ctx ret[0], setCtx[0xaaaae7011ab0], stream id[4] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.432.883 [hcom_executor.cc:45] [619427][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.432.925 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:814] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.432.955 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:616] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.432.982 [hcom_ops_kernel_info_store.cc:3423] [619427]finalize hccl kernel info store. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.002 [hcom_common.cc:85] [619427][HcomGetCurHcomCtx] hrtGetDevice deviceLogicId[2] [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.021 [plugin_manager.cc:56] [619427]hccl ops plugin finalize start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.040 [hcom_plugin.cc:90] [619427]Finalize start. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.149 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.171 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.187 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.200 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.214 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.227 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.239 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.255 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.268 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.278 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.290 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.302 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.332 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.347 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.360 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.372 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.398 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.412 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.424 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.434 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.462 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.474 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.486 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.499 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.525 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.539 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.548 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.559 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.586 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.598 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.617 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.629 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.661 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.674 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.685 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.701 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.725 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.739 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.751 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.767 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.796 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.808 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.818 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.828 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.855 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.867 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.877 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.888 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.915 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.929 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.940 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.951 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.976 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.433.993 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.003 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.016 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.041 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.057 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.069 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.082 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.107 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.124 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.136 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.154 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.168 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.183 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.194 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.204 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.215 [hcom_plugin.cc:119] [619427]Finalize: HcomDestroy success. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.229 [plugin_manager.cc:62] [619427]hccl ops plugin finalize end. ret[0] [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.434.291 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:638] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.434.313 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:695] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.325 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[0].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.346 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.359 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.370 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.381 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[1].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.391 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.404 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.415 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.427 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[2].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.439 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.450 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.465 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.477 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[3].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.488 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.499 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.512 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.528 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[4].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.541 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.554 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.568 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.583 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[5].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.596 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.608 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.636 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.648 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[6].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.661 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.671 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.683 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.695 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[7].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.708 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.719 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.732 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.746 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[8].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.761 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.772 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.782 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.792 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[9].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.804 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.818 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.831 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.843 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[10].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.854 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.864 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.878 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.893 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[11].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.904 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.915 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.925 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.940 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[12].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.953 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.965 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.978 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.434.993 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[13].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.007 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.016 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.027 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.040 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[14].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.063 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.075 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.085 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.097 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[15].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.110 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.122 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.130 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.144 [hcom_common.cc:958] [619427][Destroy][Result]hcomInfo[16].isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.157 [hcom_common.cc:961] [619427]Entry-HcomDestroy:void skip [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.170 [hcom.cc:337] [619427][Destroy][Result]hcomInfo.isHcomInit is false. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:52.435.185 [hcom.cc:340] [619427]Entry-HcomDestroy:void skip [INFO] HCCL_ADPT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.211 [mindspore/ccsrc/plugin/res_manager/ascend/hccl_adapter/hccl_adapter.cc:377] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.233 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:613] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.451 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 1, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.480 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] In used mem buf info for Mem buf allocator, is persistent : 0, stream id : 0., mem_bufs size : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.500 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 0 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.513 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 0, stream id : 0. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.527 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.544 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12d100000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200} [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.563 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61022928896 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.577 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.590 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61022928384, total count : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.604 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] stream id : 0, is persistent : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.624 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Dump state info for Mem buf allocator, is persistent : 1, stream id : 0. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.650 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block info - total count : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.667 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] {"addr_":0x12c2c0000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200} [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.685 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Mem block total size : 61022928896 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.701 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Free mem buf total size : 0, total count : 0 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.716 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] Eager free mem buf total size : 61022928384, total count : 1 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.734 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116392.00MB,"idle mem":116391.99MB,"used by event mem":0.00MB,"eager free mem":116391.99MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.764 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":116392.00MB,"idle mem":116391.99MB,"used by event mem":0.00MB,"eager free mem":116391.99MB}. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.781 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 0, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.800 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12d100000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.817 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.834 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.853 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12d100000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12d100000000,"max_addr_":0x12d100000200}. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.868 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.884 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.901 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:62] ReleaseDeviceRes] Release device resource for allocator, Mem buf allocator, is persistent : 1, stream id : 0., mem_blocks_ size : 1. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.918 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:65] ReleaseDeviceRes] Clean mem block : {"addr_":0x12c2c0000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.943 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:191] FreeDeviceMem] Max actual used memory size is 4194304 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.959 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/abstract_ascend_memory_pool_support.cc:194] FreeDeviceMem] Max peak used memory size is 1024 [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.976 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:69] ReleaseDeviceRes] Delete mem block : {"addr_":0x12c2c0000000,"size_":61022928896,"stream_id_":0,"min_addr_":0x12c2c0000000,"max_addr_":0x12c2c0000200}. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.435.990 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:74] ReleaseDeviceRes] Free mem buf size : 0. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.436.008 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:80] ReleaseDeviceRes] Eager free mem buf size : 1. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.436.027 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe88100790] : Mem buf allocator, is persistent : 1, stream id : 0. deconstruct. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.436.043 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:45] ~MemBufAllocator] MemBufAllocator[0xfffe88101080] : Mem buf allocator, is persistent : 0, stream id : 0. deconstruct. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.436.066 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:222] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device MOC memory size: 62420M MindSpore Used memory size: 58196M Used peak memory usage (without fragments): 0M Actual peak memory usage (with fragments): 4M [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.096 [callback.cpp:37]619427 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.117 [callback.cpp:44]619427 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.173 [context.cpp:72]619427 aclrtSetCurrentContext: start to execute aclrtSetCurrentContext. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.198 [api_impl.cc:3717] 619427 ContextSetCurrent: set context=aaaae7011ab0, flag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.215 [context.cpp:80]619427 aclrtSetCurrentContext: successfully execute aclrtSetCurrentContext [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.235 [stream.cpp:75]619427 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] GE(619427,python3.9):2025-04-03-09:35:52.436.256 [stream_manage_func_registry.cc:22]619427 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.267 [stream.cpp:80]619427 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.279 [api_impl.cc:1228] 619427 StreamDestroy: stream_id=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.295 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.436.310 [single_op_manager.cc:50]619427 ReleaseResource:ReleaseResource in. resource id = 0xaaaae80610a0 [INFO] GE(619427,python3.9):2025-04-03-09:35:52.436.324 [caching_allocator.cc:402]619427 TryFreeBlocks:Try free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.436.337 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:52.436.362 [npu_memory_allocator.cc:72]619427 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.376 [stream_executor.cpp:197]619427 RemoveExecutor: To remove executor by stream = 187651013873824 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.386 [acl_resource_manager.cpp:823]619427 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.398 [stream.cc:852] 619427 TearDown: stream_id=2, sqId=2, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.410 [stream.cc:938] 619427 TearDown: stream_id=2, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.427 [event.cc:1036] 619427 WaitForBusy: device_id=2, event_id=0 finished map already empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.440 [npu_driver.cc:626] 619427 EventIdFree: event id free begin, device_id=2, event_id=0, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.464 [raw_device.cc:1158] 619427 FreeEventIdFromDrv: device_id=2, event_id=0 tear down. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.477 [event.cc:61] 619427 ~Event: device_id=2, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.497 [npu_driver.cc:3803] 619427 SqCqFree: deviceId=2, tsId=0, sqId=458, cqId=458 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.549 [subscribe.cc:244] 619427 Delete: delete callback subscribe: groupId=576, threadId=281471067484064, devId=2, tsId=0, streamId=2, sqId=458, cqId=458 [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.436.553 [trs_sqcq.c:1491][ascend][curpid:619427,619588][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.568 [stream.cc:1507] 619427 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.583 [npu_driver.cc:3833] 619588 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=576, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.605 [api_impl.cc:5058] 619588 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.604 [stream_sqcq_manage.cc:451] 619427 FreeLogicCq: Return(0), threadIdentifier(281473076260896), devId(2), tsId(0), cqId(1006), isFastCq(0). [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.623 [callback.cpp:132]619588 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.632 [npu_driver.cc:6163] 619427 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=2, logicCq=1006. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.642 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.665 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.675 [npu_driver.cc:1180] 619427 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1006, drvFlag=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.677 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.711 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.722 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.734 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.760 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.772 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.784 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.797 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.810 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.822 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.833 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.849 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.861 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.872 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.883 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.894 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.909 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.922 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.934 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.947 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.957 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.968 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.436.982 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.436.997 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.009 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.020 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.031 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.043 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.053 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.070 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.081 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.092 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.106 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.120 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.135 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.148 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.161 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.172 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.181 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.191 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.206 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.218 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.228 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.240 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.253 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.264 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.278 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.292 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.304 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.314 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.325 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.335 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.349 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.359 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.379 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.389 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.401 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.411 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.425 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.438 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.446 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.457 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.471 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.484 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.500 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.514 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.525 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.536 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.546 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.558 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.573 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.584 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.597 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.609 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.619 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.634 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.646 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.658 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.671 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.700 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.715 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.725 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.737 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.750 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.764 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.779 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.790 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.802 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.814 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.828 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.843 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.852 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.862 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.874 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.885 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.902 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.914 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.925 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.937 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.947 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.437.957 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.969 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.437.981 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.004 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.015 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.028 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.039 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.050 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.064 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.076 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.090 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.103 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.113 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.125 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.137 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.147 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.156 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.169 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.180 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.192 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.206 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.217 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.230 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.242 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.252 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.264 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.277 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.288 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.304 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.314 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.327 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.339 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.353 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.366 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.377 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.388 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.397 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.408 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.423 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.436 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.447 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.458 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.470 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.482 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.496 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.507 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.519 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.534 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.549 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.564 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.579 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.592 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.603 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.621 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.634 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.647 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.658 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.669 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.679 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.691 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.701 [stream.cc:237] 619427 FreeStreamId: Free stream_id=2. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.707 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.724 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.733 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.745 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=2, cqId=2, device_id=2, tsId=0, drvFlag=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.754 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.776 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.790 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.800 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.812 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.824 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.835 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.848 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.864 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.877 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.889 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.903 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.913 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.934 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.948 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.960 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.438.971 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.982 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.438.992 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.005 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.018 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.029 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.040 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.053 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.064 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.078 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.092 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.103 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.114 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.124 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.134 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.150 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.165 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.175 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.184 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.196 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.209 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.224 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.229 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=2, cqId=2, device_id=2, tsId=0, drvFlag=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.246 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.261 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=2, cq_id=2, stream_id=2, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.269 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.283 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=2, ts_id=0, remote_stream_id=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.293 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.310 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=2, device_id=2, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.315 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.339 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.352 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.364 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.380 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.393 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.408 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.423 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.434 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.449 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.462 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.476 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.490 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.500 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.510 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.522 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.537 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.555 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.565 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.577 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.590 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.605 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.619 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.629 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.640 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.651 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.660 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.677 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.692 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.706 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.717 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.729 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.743 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.759 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.773 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.788 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.799 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.809 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.822 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.835 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.849 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.863 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.894 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.907 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.919 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.931 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.946 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.439.961 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.978 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.439.991 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.004 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.014 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.025 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.039 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.053 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.064 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.076 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.088 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.100 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.112 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.125 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.137 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.149 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.184 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.200 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.214 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.224 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.244 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.256 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.267 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.279 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.290 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.299 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.311 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.323 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.334 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.348 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.362 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.374 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.385 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.396 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.406 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.418 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.431 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.449 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.460 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.472 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.486 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.500 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.516 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.529 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.541 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.561 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.571 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.584 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.595 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.606 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.618 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.632 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.642 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.655 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.666 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.678 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.689 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.703 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.716 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.728 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.741 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.756 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.767 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.775 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.787 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.799 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.814 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.826 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.841 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.860 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.872 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.887 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.898 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.910 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.921 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.934 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.947 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.959 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.440.974 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.986 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.440.998 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.008 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.018 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.028 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.040 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.055 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.070 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.081 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.092 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.102 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.115 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.126 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.137 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.151 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.172 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.187 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.205 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.218 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.231 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.244 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.259 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.274 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.286 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.299 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.309 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.319 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.330 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.344 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.356 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.368 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.379 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.389 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.400 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.413 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.426 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.438 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.448 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.459 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.471 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.501 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.513 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.526 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.537 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.550 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.561 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.574 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.585 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.595 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.607 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.620 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.629 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.640 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.651 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.666 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.680 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.691 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.703 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.714 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.727 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.741 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.752 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.762 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.774 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.785 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.810 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.822 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.831 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.843 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.856 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.869 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.881 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.891 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.902 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.913 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.927 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.944 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.957 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.968 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.441.978 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.441.991 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.001 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.015 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.030 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.043 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.055 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.068 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.082 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.095 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.107 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.125 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.135 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.147 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.160 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.175 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.185 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.197 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.208 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.223 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.238 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.249 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.260 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.271 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.282 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.294 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.306 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.317 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.329 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.341 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.354 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.370 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.382 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.392 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.403 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.423 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.437 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.448 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.457 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.470 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.480 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.495 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.508 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.517 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.527 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.537 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.549 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.563 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.573 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.585 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.598 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.613 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.624 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.635 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.647 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.660 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.672 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.684 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.694 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.706 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.732 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.745 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.758 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.772 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.783 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.794 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.810 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.821 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.832 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.842 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.854 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.867 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.878 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.889 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.902 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.913 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.927 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.938 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.950 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.961 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.442.972 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.986 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.442.997 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.009 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.019 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.040 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.052 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.062 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.073 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.086 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.098 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.111 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.127 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.138 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.149 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.160 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.172 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.186 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.196 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.206 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.218 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.229 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.242 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.251 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.262 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.275 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.288 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.304 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.317 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.331 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.352 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.365 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.377 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.393 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.405 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.417 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.430 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.443 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.459 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.472 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.486 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.490 [stream.cpp:87]619427 aclrtDestroyStream: aclrtDestroyStream success [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.497 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.525 [callback.cpp:116]619588 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.536 [subscribe.cc:308] 619588 GetGroupIdByThreadId: can not find subscribe info, threadId=281471067484064 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.443.540 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaae80610a0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.548 [api_impl.cc:5043] 619588 ProcessReport: get groupId fail, threadIdentifier=281471067484064, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.568 [stream.cpp:75]619427 aclrtDestroyStream: start to execute aclrtDestroyStream [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.573 [callback.cpp:130]619588 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] GE(619427,python3.9):2025-04-03-09:35:52.443.591 [stream_manage_func_registry.cc:22]619427 TryCallStreamMngFunc:Stream manage func is not found, FuncType is [0] [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.609 [stream.cpp:80]619427 aclrtDestroyStream: try to call stream manage function success, action type is DESTROY_STREAM. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.625 [api_impl.cc:1228] 619427 StreamDestroy: stream_id=3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.643 [acl_resource_manager.cpp:814]619427 HandleReleaseSourceByStream: start to execute HandleReleaseSourceByStream. [INFO] DEVICE(619427,ffff16fdefa0,python3.9):2025-04-03-09:35:52.443.649 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.443.659 [single_op_manager.cc:50]619427 ReleaseResource:ReleaseResource in. resource id = 0xaaaae8191fc0 [INFO] GE(619427,python3.9):2025-04-03-09:35:52.443.696 [caching_allocator.cc:402]619427 TryFreeBlocks:Try free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:52.443.712 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:52.443.723 [npu_memory_allocator.cc:72]619427 FreeCachedMem:NpuMemoryAllocators start free cached mem. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.737 [stream_executor.cpp:197]619427 RemoveExecutor: To remove executor by stream = 187651015122880 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.751 [acl_resource_manager.cpp:823]619427 HandleReleaseSourceByStream: successfully execute HandleReleaseSourceByStream. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.769 [stream.cc:852] 619427 TearDown: stream_id=3, sqId=3, pendingNum=0, isForceRecycle_=0, flag=0, failMode=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.783 [stream.cc:938] 619427 TearDown: stream_id=3, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.795 [event.cc:1036] 619427 WaitForBusy: device_id=2, event_id=1 finished map already empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.808 [npu_driver.cc:626] 619427 EventIdFree: event id free begin, device_id=2, event_id=1, ts_id=0, remote_event_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.827 [raw_device.cc:1158] 619427 FreeEventIdFromDrv: device_id=2, event_id=1 tear down. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.844 [event.cc:61] 619427 ~Event: device_id=2, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.861 [npu_driver.cc:3803] 619427 SqCqFree: deviceId=2, tsId=0, sqId=459, cqId=459 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.895 [subscribe.cc:244] 619427 Delete: delete callback subscribe: groupId=577, threadId=281471059029920, devId=2, tsId=0, streamId=3, sqId=459, cqId=459 [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.443.902 [trs_sqcq.c:1491][ascend][curpid:619427,619589][drv][tsdrv][trsCbCqRecv]Recv warn. (devId=2; type=1; ret=16) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.914 [stream.cc:1507] 619427 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.929 [npu_driver.cc:3833] 619589 CqReportIrqWait: [drv api] halCqReportIrqWait: device_id=2, tsId=0, groupId=577, drvRetCode=16. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.950 [api_impl.cc:5058] 619589 ProcessReport: CqReportIrqWait, retCode=0x711000c [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.950 [stream_sqcq_manage.cc:451] 619427 FreeLogicCq: Return(0), threadIdentifier(281473076260896), devId(2), tsId(0), cqId(1007), isFastCq(0). [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.966 [callback.cpp:132]619589 aclrtProcessReport: wait subscribereport timeout, runtime errorCode = 507012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.443.978 [npu_driver.cc:6163] 619427 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=3, logicCq=1007. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.443.985 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.008 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.017 [npu_driver.cc:1180] 619427 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1007, drvFlag=0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.023 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.045 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.067 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.081 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.092 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.104 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.116 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.130 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.145 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.181 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.193 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.208 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.221 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.230 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.240 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.253 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.267 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.278 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.290 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.305 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.318 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.328 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.337 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.348 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.362 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.376 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.386 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.405 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.419 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.432 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.442 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.453 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.467 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.477 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.487 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.500 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.512 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.523 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.532 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.542 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.554 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.565 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.576 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.586 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.596 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.608 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.620 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.632 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.642 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.656 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.671 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.682 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.699 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.713 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.726 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.737 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.747 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.760 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.770 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.780 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.790 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.803 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.818 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.830 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.840 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.851 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.860 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.870 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.883 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.897 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.910 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.920 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.929 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.941 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.444.953 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.962 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.444.975 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.018 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.029 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.040 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.049 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.060 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.070 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.081 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.092 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.102 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.113 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.122 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.133 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.143 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.154 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.166 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.177 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.188 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.200 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.212 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.220 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.230 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.242 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.253 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.264 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.282 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.292 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.304 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.315 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.327 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.337 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.351 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.364 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.376 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.385 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.395 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.407 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.418 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.427 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.438 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.448 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.459 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.470 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.482 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.492 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.503 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.513 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.523 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.533 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.544 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.560 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.573 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.587 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.601 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.612 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.627 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.637 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.646 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.657 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.666 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.676 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.687 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.697 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.708 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.719 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.731 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.741 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.751 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.756 [stream.cc:237] 619427 FreeStreamId: Free stream_id=3. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.761 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.778 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.786 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.798 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=3, cqId=3, device_id=2, tsId=0, drvFlag=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.806 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.826 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.845 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.856 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.865 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.875 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.885 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.898 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.909 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.919 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.929 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.940 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.951 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.963 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.445.973 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.983 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.445.999 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.012 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.024 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.033 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.042 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.052 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.063 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.073 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.084 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.094 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.105 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.122 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.135 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.144 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.154 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.164 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.175 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.186 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.199 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.211 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.221 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.231 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.241 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.248 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=3, cqId=3, device_id=2, tsId=0, drvFlag=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.251 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.270 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=3, cq_id=3, stream_id=3, is_sq_need_release=1, drvFlag=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.277 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.290 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=3, ts_id=0, remote_stream_id=0 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.300 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.317 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=3, device_id=2, tsId=0, streamFlag=1536 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.323 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.345 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.355 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.366 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.378 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.395 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.407 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.419 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.430 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.439 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.450 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.462 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.474 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.483 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.493 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.502 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.513 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.522 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.531 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.542 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.555 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.567 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.577 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.588 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.597 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.607 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.616 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.628 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.640 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.652 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.668 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.680 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.690 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.700 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.712 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.724 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.735 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.745 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.757 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.768 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.779 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.789 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.799 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.812 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.822 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.834 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.847 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.858 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.867 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.877 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.889 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.902 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.911 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.921 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.932 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.948 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.959 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.968 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.446.978 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.446.988 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.000 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.010 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.020 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.031 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.041 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.053 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.065 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.076 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.089 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.101 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.114 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.125 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.135 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.146 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.156 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.167 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.177 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.187 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.198 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.210 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.224 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.234 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.244 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.256 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.265 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.275 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.286 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.296 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.308 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.318 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.328 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.339 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.348 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.358 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.369 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.379 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.390 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.402 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.414 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.424 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.434 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.447 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.457 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.468 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.487 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.498 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.510 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.519 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.528 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.538 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.548 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.559 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.569 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.580 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.589 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.601 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.612 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.623 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.634 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.645 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.656 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.666 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.676 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.688 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.700 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.713 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.724 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.735 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.747 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.765 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.777 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.789 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.801 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.812 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.824 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.836 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.847 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.859 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.870 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.880 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.891 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.903 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.915 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.926 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.936 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.945 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.956 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.967 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.447.978 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.447.990 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.005 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.017 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.028 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.027 [event.cc:61] 619427 ~Event: device_id=2, event deconstruct, recordSize=0, waitSize=0, idSize=0 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.040 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.076 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.087 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.087 [stream.cpp:87]619427 aclrtDestroyStream: aclrtDestroyStream success [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.098 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.119 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.130 [callback.cpp:116]619589 aclrtProcessReport: start to execute aclrtProcessReport, timeout is 500ms. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.131 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/ascend_stream_manager.cc:137] UnRegCallback] Unregister callback thread, stream : 0xaaaae8191fc0. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.143 [subscribe.cc:308] 619589 GetGroupIdByThreadId: can not find subscribe info, threadId=281471059029920 [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.175 [api_impl.cc:5043] 619589 ProcessReport: get groupId fail, threadIdentifier=281471059029920, reCode=0x7160004 [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.180 [device.cpp:183]619427 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.187 [callback.cpp:130]619589 aclrtProcessReport: no subscribereport info, runtime errorCode = 107012 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.204 [api_impl.cc:3419] 619427 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.225 [runtime.cc:3444] 619427 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x5, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.448.238 [api_impl.cc:3449] 619427 DeviceReset: Succ, userDeviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.247 [device.cpp:189]619427 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 2. [INFO] DEVICE(619427,ffff167cefa0,python3.9):2025-04-03-09:35:52.448.252 [mindspore/ccsrc/plugin/res_manager/ascend/stream_manager/callback_thread.cc:42] callback_thread_func] Exit callback thread loop. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.272 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:206] ReleaseDeviceRes] Ascend finalize end [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.325 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:948] SplitAndDumpLog] The dynamic memory pool[DefaultAscendMemoryPool] stat info : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}, actual peak used mem:4M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.357 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:430] ReleaseDeviceRes] Release device resource for DefaultAscendMemoryPool : {"in used mem":0.00MB,"peak used mem":0.00MB,"alloc mem":0.00MB,"idle mem":0.00MB,"used by event mem":0.00MB,"eager free mem":0.00MB}. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.376 [mindspore/ccsrc/plugin/res_manager/ascend/mem_manager/ascend_memory_adapter.cc:217] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.390 [callback.cpp:37]619427 aclrtSetExceptionInfoCallback: start to execute aclrtSetExceptionInfoCallback. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.448.426 [callback.cpp:44]619427 aclrtSetExceptionInfoCallback: successfully execute aclrtSetExceptionInfoCallback [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.448 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:118] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.448.466 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel _npu_log begins the destruction process. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.533.836 [awatchdog_monitor.c:160](tid:619463) thread 619647 not exist [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.533.873 [awatchdog_monitor.c:160](tid:619463) thread 619648 not exist [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.533.896 [awatchdog_monitor.c:160](tid:619463) thread 619649 not exist [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:52.533.919 [awatchdog_monitor.c:160](tid:619463) thread 619650 not exist [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.748.716 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.748.745 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.748.759 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:52.748.773 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.748.787 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.748.853 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.171 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.180 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.182 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.182 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=33; gid=44; tid=5; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.184 [queue_client_comm.c:114][ascend][curpid:619427,619635][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:52.964.200 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.290 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.310 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.313 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.332 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.340 [tensor_data_transfer.cpp:1054]619639 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.376 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.346 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.382 [tensor_data_transfer.cpp:1054]619645 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_histogram_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.402 [tensor_data_transfer.cpp:1056]619639 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.411 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.412 [npu_driver.cc:4338] 619635 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.419 [tensor_data_transfer.cpp:1056]619645 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.416 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.432 [api_impl.cc:6121] 619639 MemQueuePeek: Start to peek queue on devId 2, qid is 2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.440 [tensor_data_transfer.cpp:1054]619643 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_scalar_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.442 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.445 [tensor_data_transfer.cpp:612]619635 acltdtReceiveTensorV2: queue is empty, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.448 [api_impl.cc:6121] 619645 MemQueuePeek: Start to peek queue on devId 2, qid is 5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.452 [tensor_data_transfer.cpp:1054]619637 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.466 [npu_driver.cc:4330] 619639 MemQueuePeek: peek mem queue, deviceId=2, qid=2, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.473 [tensor_data_transfer.cpp:1056]619643 acltdtReceiveTensor: new process, use queue process [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.480 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.487 [npu_driver.cc:4330] 619645 MemQueuePeek: peek mem queue, deviceId=2, qid=5, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.490 [tensor_data_transfer.cpp:1056]619637 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.514 [api_impl.cc:6121] 619643 MemQueuePeek: Start to peek queue on devId 2, qid is 4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.517 [tensor_data_transfer.cpp:1054]619641 acltdtReceiveTensor: start to execute acltdtReceiveTensor, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.525 [api_impl.cc:6121] 619637 MemQueuePeek: Start to peek queue on devId 2, qid is 1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.527 [npu_driver.cc:4330] 619643 MemQueuePeek: peek mem queue, deviceId=2, qid=4, timeout=800. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.530 [tensor_data_transfer.cpp:1056]619641 acltdtReceiveTensor: new process, use queue process [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.536 [npu_driver.cc:4330] 619637 MemQueuePeek: peek mem queue, deviceId=2, qid=1, timeout=800. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.542 [api_impl.cc:6121] 619641 MemQueuePeek: Start to peek queue on devId 2, qid is 3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.560 [npu_driver.cc:4330] 619641 MemQueuePeek: peek mem queue, deviceId=2, qid=3, timeout=800. [INFO] DEVICE(619427,fffeeccdefa0,python3.9):2025-04-03-09:35:52.964.578 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.964.726 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is _npu_log [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.754 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:52.964.772 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=0. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:52.965.189 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is _npu_log [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.965.228 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:52.965.252 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [WARNING] DRV(619427,python3.9):2025-04-03-09:35:53.796.177 [queue_client_comm.c:114][ascend][curpid:619427,619645][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=29; gid=44; tid=1; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:53.796.178 [queue_client_comm.c:114][ascend][curpid:619427,619639][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=28; gid=44; tid=0; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:53.796.183 [queue_client_comm.c:114][ascend][curpid:619427,619637][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=31; gid=44; tid=3; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:53.796.182 [queue_client_comm.c:114][ascend][curpid:619427,619641][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=32; gid=44; tid=4; timeout=800ms) [WARNING] DRV(619427,python3.9):2025-04-03-09:35:53.796.185 [queue_client_comm.c:114][ascend][curpid:619427,619643][drv][queuemng][QueueWaitEvent]HalEschedWaitEvent invalid. (ret=16; event_id=30; gid=44; tid=2; timeout=800ms) [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.277 [npu_driver.cc:4338] 619645 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.292 [tensor_data_transfer.cpp:612]619645 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_histogram_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.312 [npu_driver.cc:4338] 619639 MemQueuePeek: queue empty [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.335 [npu_driver.cc:4338] 619637 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.334 [tensor_data_transfer.cpp:612]619639 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_summary [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.352 [tensor_data_transfer.cpp:612]619637 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_tensor_dump [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.364 [npu_driver.cc:4338] 619641 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.380 [tensor_data_transfer.cpp:612]619641 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_image_summary [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.401 [npu_driver.cc:4338] 619643 MemQueuePeek: queue empty [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.425 [tensor_data_transfer.cpp:612]619643 acltdtReceiveTensorV2: queue is empty, device is 2, name is ms_scalar_summary [INFO] DEVICE(619427,fffecd7aefa0,python3.9):2025-04-03-09:35:53.796.465 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619427,fffee4c29fa0,python3.9):2025-04-03-09:35:53.796.500 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619427,fffecf7eefa0,python3.9):2025-04-03-09:35:53.796.522 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619427,fffece7cefa0,python3.9):2025-04-03-09:35:53.796.534 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] DEVICE(619427,fffee5c49fa0,python3.9):2025-04-03-09:35:53.796.506 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:152] ~AclDatasetInfo] AcltdtDestroyDataset succeeded. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.796.745 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is ms_histogram_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.833 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.796.853 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=5. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.797.283 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is ms_histogram_summary [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.797.372 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.797.418 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.797.475 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is ms_scalar_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.797.497 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 4. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.797.511 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=4. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.797.860 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is ms_scalar_summary [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.797.893 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.797.916 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.797.967 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is ms_image_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.797.988 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 3. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.798.001 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=3. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.798.352 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is ms_image_summary [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.798.413 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.798.435 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.798.503 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is ms_tensor_summary [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.798.522 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.798.536 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.798.885 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is ms_tensor_summary [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.798.917 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.h:252] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.798.939 [mindspore/ccsrc/plugin/res_manager/ascend/mbuf_manager/mbuf_receive_manager.cc:325] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.798.990 [tensor_data_transfer.cpp:973]619427 acltdtDestroyChannel: start to acltdtDestroyChannel, device is 2, name is ms_tensor_dump [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.799.010 [api_impl.cc:6065] 619427 MemQueueDestroy: Start to destroy queue on devId 2, qid is 1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.799.023 [npu_driver.cc:3989] 619427 MemQueueDestroy: deviceId=2, qid=1. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.799.438 [tensor_data_transfer.cpp:977]619427 acltdtDestroyChannel: acltdtDestroyChannel success, device is 2, name is ms_tensor_dump [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.799.472 [device.cpp:183]619427 aclrtResetDevice: start to execute aclrtResetDevice, deviceId = 2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.799.498 [api_impl.cc:3419] 619427 DeviceReset: reset device, device_id=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.799.516 [runtime.cc:3444] 619427 PrimaryContextRelease: Context TryDecRef, ts_id=0, count=0x4, reset:0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.799.536 [api_impl.cc:3449] 619427 DeviceReset: Succ, userDeviceId=2. [INFO] ASCENDCL(619427,python3.9):2025-04-03-09:35:53.799.547 [device.cpp:189]619427 aclrtResetDevice: successfully execute aclrtResetDevice, reset device 2. [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.576 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:136] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.615 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:401] ClearDeviceContexts] Release device CPU_0 [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.669 [mindspore/ccsrc/pybind_api/resource/manager.cc:167] ClearResPart2] End clear device context. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.689 [mindspore/ccsrc/pybind_api/resource/manager.cc:169] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.712 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:26] ClearRuntimeResource] Release device Ascend_2 [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.733 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:178] ReleaseDeviceRes] Ascend finalize start [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.772 [mindspore/ccsrc/pybind_api/resource/manager.cc:171] ClearResPart2] End clear kernel runtime. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.792 [mindspore/ccsrc/pybind_api/resource/manager.cc:173] ClearResPart2] Start clear CollectiveManager... [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.799.831 [mindspore/ccsrc/distributed/collective/collective_manager.cc:539] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.081 [mindspore/ccsrc/distributed/collective/collective_manager.cc:513] operator()] Start finalizing host communication lib. [INFO] DEVICE(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.121 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:64] Finish] Begin to finish the cpu collective node. [INFO] DEVICE(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.142 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_node.cc:69] Finish] The cpu collective node has been finished successfully. [INFO] PS(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.171 [mindspore/ccsrc/ps/core/ps_worker_node.cc:45] Stop] Stop worker node! [INFO] PS(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.204 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:176] Stop] Stop tcp client! [INFO] PS(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.237 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] DISTRIBUTED(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.262 [mindspore/ccsrc/distributed/collective/collective_manager.cc:517] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.281 [mindspore/ccsrc/distributed/collective/collective_manager.cc:522] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(619427,fffee5439fa0,python3.9):2025-04-03-09:35:53.800.301 [mindspore/ccsrc/distributed/collective/collective_manager.cc:526] operator()] End finalizing device communication lib. [INFO] PS(619427,ffff177eefa0,python3.9):2025-04-03-09:35:53.800.297 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success! [INFO] PS(619427,ffff17ffefa0,python3.9):2025-04-03-09:35:53.800.302 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success! [INFO] DISTRIBUTED(619427,fffea9fbefa0,python3.9):2025-04-03-09:35:53.800.360 [mindspore/ccsrc/distributed/collective/collective_manager.cc:960] RunInitCommTasks] Initialize communciator thread is stopped. [INFO] DISTRIBUTED(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.454 [mindspore/ccsrc/distributed/collective/collective_manager.cc:547] Finalize] End finalize collective manager. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.486 [mindspore/ccsrc/pybind_api/resource/manager.cc:176] ClearResPart2] End clear CollectiveManager. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.505 [mindspore/ccsrc/pybind_api/resource/manager.cc:178] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.528 [mindspore/ccsrc/pybind_api/resource/manager.cc:180] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.546 [mindspore/ccsrc/pybind_api/resource/manager.cc:182] ClearResPart2] Start clear AnalysisContext... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.562 [mindspore/ccsrc/pybind_api/resource/manager.cc:184] ClearResPart2] End clear AnalysisContext... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.580 [mindspore/ccsrc/pybind_api/resource/manager.cc:186] ClearResPart2] Start clear AnalysisSchedule... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.756 [mindspore/ccsrc/pybind_api/resource/manager.cc:188] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.803 [mindspore/ccsrc/debug/debugger/debugger.cc:149] Reset] Release Debugger resource. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.860 [mindspore/ccsrc/pybind_api/resource/manager.cc:200] ClearResPart3] Start clear ClearObjectCache... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.881 [mindspore/ccsrc/pybind_api/resource/manager.cc:202] ClearResPart3] End clear ClearObjectCache... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.896 [mindspore/ccsrc/pybind_api/resource/manager.cc:204] ClearResPart3] Start clear Parser... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.919 [mindspore/ccsrc/pybind_api/resource/manager.cc:206] ClearResPart3] End clear Parser... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.800.937 [mindspore/ccsrc/pybind_api/resource/manager.cc:208] ClearResPart3] Start ClearTraceStack... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.016 [mindspore/ccsrc/pybind_api/resource/manager.cc:210] ClearResPart3] End ClearTraceStack... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.036 [mindspore/ccsrc/pybind_api/resource/manager.cc:212] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.053 [mindspore/ccsrc/pybind_api/resource/manager.cc:214] ClearResPart3] End clear InterpretNodeRecorder... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.071 [mindspore/ccsrc/pybind_api/resource/manager.cc:216] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.111 [mindspore/ccsrc/pybind_api/resource/manager.cc:218] ClearResPart3] End clear parallel::entire_costgraph... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.130 [mindspore/ccsrc/pybind_api/resource/manager.cc:220] ClearResPart3] Start clear ProtobufLibrary... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.466 [mindspore/ccsrc/pybind_api/resource/manager.cc:222] ClearResPart3] End clear ProtobufLibrary... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.487 [mindspore/ccsrc/pybind_api/resource/manager.cc:224] ClearResPart3] Start clear ParserDefaultObjects ... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.520 [mindspore/ccsrc/pybind_api/resource/manager.cc:226] ClearResPart3] End clear ParserDefaultObjects... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.540 [mindspore/ccsrc/pybind_api/resource/manager.cc:231] ClearResPart3] Start clear python_adapter... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.556 [mindspore/ccsrc/pybind_api/resource/manager.cc:233] ClearResPart3] End clear python_adapter. [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.575 [mindspore/ccsrc/pybind_api/resource/manager.cc:237] ClearSingleton] Start clear singleton... [INFO] COMMON(619427,ffff8eb97020,python3.9):2025-04-03-09:35:53.801.835 [mindspore/ccsrc/pybind_api/resource/manager.cc:255] ClearSingleton] End clear singleton. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:53.870.549 [awatchdog_monitor.c:160](tid:619463) thread 619651 not exist [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.964.759 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:53.964.828 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:53.964.846 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:53.964.859 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.964.888 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:53.965.017 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:54.344.765 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12d100000000. [INFO] PRE_ACT(619427,ffff8eb97020,python3.9):2025-04-03-09:35:54.344.930 [mindspore/ccsrc/memory/mem_pool/abstract_dynamic_mem_pool.cc:591] DoFreeTensorMem] Free tensor mem failed, can not find address : 0x12c2c0000000. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.782.234 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.782.757 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.782.793 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.413 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.437 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.454 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.473 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.498 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.783.515 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.000 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.022 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.264 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.286 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.874 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.896 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.784.985 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.005 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.019 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.032 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.045 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.061 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.073 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.086 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.102 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.115 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.127 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.140 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.154 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.167 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.183 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.221 [heartbeat.cc:145] [619427][DeInit]heartbeat deinit begin [INFO] HCCL(619427,python3.9):2025-04-03-09:35:54.785.237 [heartbeat.cc:183] [619427][DeInit]heartbeat deinit end [INFO] DEVICE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:54.785.789 [mindspore/ccsrc/plugin/res_manager/ascend/collective/hccl_watch_dog_thread.cc:67] ~HcclWatchDogHandler] HcclWatchDogHandler thread exit, rank id: 2, group name: hccl_world_group [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:54.785.902 [hash_data.cpp:63] >>> (tid:619427) HashData uninitialize success [INFO] PROFILING(619427,python3.9):2025-04-03-09:35:54.785.929 [msprof_reporter_mgr.cpp:270] >>> (tid:619427) The reporter isn't started, don't need to be stopped. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:54.788.443 [api_impl.cc:2003] 619427 DevFree: device free mem=0x12c041200000 [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:54.789.339 [mindspore/core/utils/temp_file_manager.cc:59] CleanTempFiles] Start delete temp files. [INFO] CORE(619427,ffff8eb97020,python3.9):2025-04-03-09:35:54.789.371 [mindspore/core/utils/temp_file_manager.cc:63] CleanTempFiles] End delete temp files. [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:54.789.604 [adx_dump_record.cpp:229][tid:619427] start to dump uninit [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:54.789.634 [adx_dump_record.cpp:239][tid:619427] dump uninit success [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:54.789.676 [adx_dump_record.cpp:674][tid:619633] exit record file thread [INFO] GE(619427,python3.9):2025-04-03-09:35:55.107.236 [operator_factory_impl.cc:392]619427 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.410 [operator_factory_impl.cc:392]619427 ReleaseRegInfo:Release ops proto reg info success. [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.477 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.496 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.508 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.523 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.535 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.547 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.560 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.147.575 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.180.740 [runtime.cc:5239] 619586 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:55.180.787 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:55.180.805 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] TDT(619427,python3.9):2025-04-03-09:35:55.180.817 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619586] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.180.835 [runtime.cc:3077] 619586 GetHdcConctStatus: tsdGetHdcConctStatus success, userDeviceId=2, devId=2, tdt status=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.180.941 [npu_driver.cc:551] 619586 ReadFaultEvent: read fault event, device_id=-1, filterFlag=9, event_id=2162263040, timeout=1000ms. [INFO] GE(619427,python3.9):2025-04-03-09:35:55.319.979 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.320.268 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] GE(619427,python3.9):2025-04-03-09:35:55.320.327 [opsproto_manager.cc:66]619427 Finalize:close opsprotomanager handler success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.516.608 [api_impl.cc:7151] 619427 UnmapMem: device free mem=0x12c2c0000000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.516.861 [api_impl.cc:7135] 619427 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.524.755 [api_impl.cc:7151] 619427 UnmapMem: device free mem=0x12d100000000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.524.924 [api_impl.cc:7135] 619427 FreePhysical: Start to FreePhysical [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.532.416 [api_impl.cc:7121] 619427 ReleaseMemAddress: device free mem=0x12c2c0000000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.536.099 [api_impl.cc:7121] 619427 ReleaseMemAddress: device free mem=0x12d100000000 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:55.544.079 [osal.cc:321] 619427 ThreadExit: ThreadExit start. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:56.396.740 [stars_engine.cc:1833] 619586 MonitoringRun: g_hbmRasCheckDeviceId has change to -1, dev_id=-1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:56.396.800 [stars_engine.cc:1836] 619586 MonitoringRun: Starts Monitor thread has finished, ts_id=0. [INFO] PS(619427,ffff8eb97020,python3.9):2025-04-03-09:35:56.397.054 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:232] Stop] Stop tcp server! [INFO] GE(619427,python3.9):2025-04-03-09:35:56.466.166 [ops_kernel_builder_registry.cc:71]619427 ~OpsKernelBuilderRegistrar:Unregister kernel lib name = ops_kernel_info_hccl [INFO] GE(619427,python3.9):2025-04-03-09:35:56.466.238 [ops_kernel_builder_registry.cc:41]619427 Unregister:OpsKernelBuilder of ops_kernel_info_hccl is unregistered [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:57.444.212 [adx_server_manager.cpp:236][tid:619634] server manager stop [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:57.852.427 [adx_hdc_epoll.cpp:57][tid:619427] start to close hdc epoll [INFO] GE(619427,python3.9):2025-04-03-09:35:57.930.159 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaeccc4c60]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.935.655 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaeccc9290]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.938.783 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaeccf3710]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.943.852 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaeccc62a0]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.945.396 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaee0ff890]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.946.885 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaee127880]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.948.194 [op_impl_registry_holder_manager.cc:31]619427 CloseHandle:start close handle, handle[0xaaaaee1eaf00]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.967.612 [mem_manager.cc:118]619427 Finalize:MemManager Finalize. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.967.654 [session_scope_mem_allocator.cc:73]619427 FreeAllMemory:Free all memory [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.085 [session_scope_mem_allocator.cc:73]619427 FreeAllMemory:Free all memory [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.116 [caching_allocator.cc:474]619427 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.148 [caching_allocator.cc:391]619427 FreeBlocks:Free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.163 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.201 [caching_allocator.cc:422]619427 FreeBlockBins:Free block bins. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.213 [caching_allocator.cc:474]619427 PrintStatics:Called counts[malloc: 0 free: 0]. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.225 [caching_allocator.cc:391]619427 FreeBlocks:Free blocks. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.233 [caching_allocator.cc:358]619427 FreeCachedBlocks:Free cached blocks [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.242 [caching_allocator.cc:422]619427 FreeBlockBins:Free block bins. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.268 [mem_manager.cc:145]619427 ReleaseResource:MemManager ReleaseResource device id = 0. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.285 [rts_caching_mem_allocator.cc:150]619427 ReleaseResource:RtsAllocatorManager ReleaseResource device id:0. [INFO] GE(619427,python3.9):2025-04-03-09:35:57.970.335 [rts_caching_mem_allocator.cc:136]619427 Finalize:RtsAllocatorManager Finalize. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.241 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.314 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.325 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.337 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.347 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.369 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.383 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.392 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.401 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.412 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.421 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.432 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.441 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.450 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.459 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] HCCL(619427,python3.9):2025-04-03-09:35:58.060.468 [network_manager.cc:875] [619427]destroy all nic/vnic. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.567 [runtime.cc:1570] 619427 ~Runtime: deconstruct runtime [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.606 [runtime.cc:1577] 619427 ~Runtime: wait monitor success, use=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.860 [raw_device.cc:262] 619427 SetDevProfStatus: devProfStatus_=0, profConfigType=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.896 [engine.cc:188] 619427 SubmitTaskNormal: stream_id=0,task_id=5,task_type=83 (DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.920 [ringbuffer_maintain_task.cc:83] 619427 ConstructSqeForRingBufferMaintainTask: RingBufferMaintainTask stream_id:0, task_id:5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.935 [stream.cc:1400] 619427 AddTaskToList: public task, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), head=5, tail=6 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.949 [stream.cc:2595] 619427 StarsAddTaskToStream: single-operator flow stream, stream_id=0, task_id=5, type=83(DEVICE_RINGBUFFER_CONTROL), head=5, tail=6, sqe_num=1, lastTail=5, device_id=2 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.065.988 [npu_driver.cc:751] 619427 SqTaskSend: device_id=2, ts_id=0, sq_id=0, sqe_num=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.002 [stars_engine.cc:893] 619427 SendTask: Sq task send finished, device_id=2, ts_id=0, sq_id=0, cq_id=0, stream_id=0, task_id=5, task_type=83(DEVICE_RINGBUFFER_CONTROL). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.019 [stream.cc:1494] 619427 IsNeedSendFlipTask: IsNeedSendFlipTask, preTaskId=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.037 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.052 [stream.cc:2886] 619427 StarsWaitForTask: No.0 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.065 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.077 [stream.cc:2886] 619427 StarsWaitForTask: No.1 stream_id=0, taskId=5, currentId=4 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.087 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.103 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.118 [task.cc:185] 619427 GetTask: this stream no taskRes, stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.133 [logger.cc:152] 619427 TaskFinished: device_id=2, stream_id=0, task_id=5, task_type=83 (DEVICE_RINGBUFFER_CONTROL), task_finish_num=16 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.153 [stars_engine.cc:437] 619427 FinishedTaskReclaim: recycle task, stream_id=0, limited=0, taskHead=5, sqHead=6, endTaskId=5, recycleTaskId=5. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.167 [stream.cc:2886] 619427 StarsWaitForTask: No.2 stream_id=0, taskId=5, currentId=5 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.182 [stars_engine.cc:524] 619427 TaskReclaim: Travel all streams, num=2. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.194 [npu_driver.cc:5547] 619427 GetSqHead: deviceId=2, tsId=0, sqId=0, head=6. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.211 [stream.cc:852] 619427 TearDown: stream_id=0, sqId=0, pendingNum=0, isForceRecycle_=1, flag=1, failMode=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.226 [stream.cc:938] 619427 TearDown: stream_id=0, delay recycle task num=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.066.237 [stream.cc:1507] 619427 IsNeedSendFlipTask: profiling TrackProfFlag closed=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.698 [stream_sqcq_manage.cc:451] 619427 FreeLogicCq: Return(0), threadIdentifier(281473076260896), devId(2), tsId(0), cqId(1005), isFastCq(0). [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.749 [npu_driver.cc:6163] 619427 StreamUnBindLogicCq: success: deviceId=2, tsId=0, streamId=0, logicCq=1005. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.803 [npu_driver.cc:1180] 619427 LogicCqFree: Succ:device_id=2,ts_id=0,type=2,logic_cq_id=1005, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.819 [stream.cc:237] 619427 FreeStreamId: Free stream_id=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.832 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.069.845 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=0, cqId=0, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.475 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=0, cqId=0, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.493 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=0, cq_id=0, stream_id=0, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.508 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=0, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.538 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=0, device_id=2, tsId=0, streamFlag=192 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.607 [stars_engine.cc:206] 619587 RecycleThreadRun: RecycleThreadRun thread leave. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.750 [stars_engine.cc:191] 619427 DestroyRecycleThread: Stars engine joined recycle thread OK. [WARNING] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.765 [osal.cc:225] 619427 Join: mmJoinTask failed, thread=281471421505440 does not exist! [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.775 [stars_engine.cc:150] 619427 Stop: Startsengine joined monitor thread OK. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.797 [runtime.cc:5239] 619427 GetUserDevIdByDeviceId: No need to convert deviceId, userDevId=2, deviceId=2. [INFO] TDT(619427,python3.9):2025-04-03-09:35:58.070.814 [client_manager.cpp:635][ChangeUserDeviceIdToLogicDeviceId][tid:619427] [TsdClient] g_userDeviceInfo is empty. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.828 [runtime.cc:3003] 619427 StopAicpuExecutor: TsdClose success, userDeviceId=2, devId=2, tdt status=0 [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.070.846 [atrace_api.c:118](tid:619427) AtraceDestroy start. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.070.864 [tracer_mgr_operate.c:248](tid:619427) destroy object RUNTIME_ATRACE_DEV2_TS0, exitSave(false). [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.070.882 [atrace_api.c:120](tid:619427) AtraceDestroy end. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.901 [stream.cc:237] 619427 FreeStreamId: Free stream_id=1. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.913 [stream_sqcq_manage.cc:216] 619427 DeAllocStreamSqCq: deviceId=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.070.923 [npu_driver.cc:966] 619427 NormalSqCqFree: Normal sqcq free, sqId=1, cqId=1, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.071.133 [npu_driver.cc:974] 619427 NormalSqCqFree: Normal sqcq free success, sqId=1, cqId=1, device_id=2, tsId=0, drvFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.071.147 [stream_sqcq_manage.cc:228] 619427 DeAllocStreamSqCq: [SqCqManage]success to release sq, sq_id=1, cq_id=1, stream_id=1, is_sq_need_release=1, drvFlag=0. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.071.160 [npu_driver.cc:432] 619427 StreamIdFree: stream id free begin, device_id=2, stream_id=1, ts_id=0, remote_stream_id=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.071.173 [npu_driver.cc:441] 619427 StreamIdFree: Stream id free success, stream_id=1, device_id=2, tsId=0, streamFlag=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.081.852 [stars_engine.cc:58] 619427 ~StarsEngine: Destructor. [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.081.884 [engine.cc:84] 619427 ~Engine: Destructor. [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.742 [trs_dev_drv.c:370][ascend][curpid:619427,619427][drv][tsdrv][drvDeviceCloseComm]Close success. (devid=2; fd=36; mmapFd=37) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.770 [devmm_svm.c:2238][ascend][curpid:619427,619427][drv][devmm][drvMemDeviceCloseInner]DrvMemDeviceClose. (devid=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.787 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]SVM_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=2965372928; alloc_cnt=7; free_cnt=7) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.806 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4096; alloc_cnt=2; free_cnt=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.820 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]HOST_MEM Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=11341824; alloc_cnt=3; free_cnt=3) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.848 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]HOST_MEM Cached_size:12582912Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.861 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev2 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=94208; alloc_cnt=10; free_cnt=10) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.878 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev2 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=135168; alloced_peak_size=135168; alloc_cnt=1; free_cnt=0) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.894 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_HBM dev2 Cached_size:1961984Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.907 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev2 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=16384; alloc_cnt=1; free_cnt=1) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.923 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_DDR dev2 Cached_size:2097152Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.936 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev2 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=1343488; alloc_cnt=8; free_cnt=8) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.952 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_SMALL_P2P_HBM dev2 Cached_size:2097152Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.967 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev2 Mem stats (Bytes). (module_name=UNKNOWN; module_id=0; current_alloced_size=0; alloced_peak_size=2097152; alloc_cnt=1; free_cnt=1) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.982 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev2 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=20971520; alloc_cnt=8; free_cnt=8) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.082.996 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev2 Mem stats (Bytes). (module_name=APP; module_id=33; current_alloced_size=0; alloced_peak_size=429916160; alloc_cnt=3; free_cnt=3) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.011 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_HBM dev2 Cached_size:23068672Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.023 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev2 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=6291456; alloc_cnt=2; free_cnt=2) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.038 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_DDR dev2 Cached_size:6291456Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.050 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev2 Mem stats (Bytes). (module_name=HCCL; module_id=3; current_alloced_size=0; alloced_peak_size=4194304; alloc_cnt=1; free_cnt=1) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.069 [svm_mem_statistics.c:140][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev2 Mem stats (Bytes). (module_name=RUNTIME; module_id=7; current_alloced_size=0; alloced_peak_size=12582912; alloc_cnt=5; free_cnt=5) [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.083.084 [svm_mem_statistics.c:146][ascend][curpid:619427,619427][drv][devmm][_svm_mem_stats_show]MEM_DEV_HUGE_P2P_HBM dev2 Cached_size:16777216Bytes [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.155.973 [drv_log_user.c:623][ascend][curpid:619427,619427][drv][devmm][share_log_read_in_single_module]Close logical_device. (logical_devid=2; phyid=2; vfid=0; hostpid=619427; devpid=6117) [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.156.014 [npu_driver.cc:3045] 619427 DeviceClose: Close device success, device_id=2, tsId=0 [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.156.030 [raw_device.cc:125] 619427 ~Device: deconstruct device [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.264.465 [plugin_old_arch.cc:25] 619427 DestructorRuntimeImpl: RuntimeImpl destructor success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.264.499 [runtime_keeper.cc:167] 619427 DestoryRuntimeImpl: Destory Runtime success [INFO] RUNTIME(619427,python3.9):2025-04-03-09:35:58.265.500 [task_fail_callback_data_manager.cc:63] 619427 ~TaskFailCallBackManager: Destructor. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.351.848 [stacktrace_signal.c:188](tid:619427) unregister all signo sigaction, can not capture signal. [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.352.593 [atrace_client_communication.c:141](tid:619427) send [end] msg. [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.354.477 [hdc_core.c:1982][ascend][curpid:619427,619427][drv][hdc][drvHdcSetSessionReference]Get pid number. (session=4287; pid=619427) [INFO] IDEDD(619427,python3.9):2025-04-03-09:35:58.354.508 [hdc_api.cpp:671][tid:619427] connect succ, peer_node: 0, peer_devid: 2 [INFO] DRV(619427,python3.9):2025-04-03-09:35:58.354.675 [hdc_client.c:417][ascend][curpid:619427,619427][drv][hdc][drvHdcClientSessionClose]Destroy client session. (sock=4287) [INFO] ATRACE(619427,python3.9):2025-04-03-09:35:58.354.728 [atrace_client_communication.c:164](tid:619427) send [end] msg success.